-- Logs begin at Tue 2022-06-21 14:56:26 UTC, end at Tue 2022-06-21 16:17:55 UTC. -- Jun 21 15:39:07.358309 dv-cinder-840678-13-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 21 15:39:07.368673 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 21 15:39:07 2022] *** Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: nodename: dv-cinder-840678-13-iscsi Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: machine: x86_64 Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: clock source: unix Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: pcre jit disabled Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: detected number of CPU cores: 4 Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: current working directory: / Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: detected binary path: /usr/bin/uwsgi-core Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: your processes number limit is 47680 Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: your memory page size is 4096 bytes Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: detected max file descriptor number: 2048 Jun 21 15:39:07.371346 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: lock engine: pthread robust mutexes Jun 21 15:39:07.372445 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: thunder lock: enabled Jun 21 15:39:07.372445 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 21 15:39:07.372445 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 21 15:39:07.416249 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: Python main interpreter initialized at 0x55f698be59c0 Jun 21 15:39:07.416249 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: python threads support enabled Jun 21 15:39:07.416249 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: your server socket listen backlog is limited to 100 connections Jun 21 15:39:07.416249 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: your mercy for graceful operations on workers is 90 seconds Jun 21 15:39:07.416761 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: mapped 403077 bytes (393 KB) for 2 cores Jun 21 15:39:07.416761 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: *** Operational MODE: preforking *** Jun 21 15:39:07.417010 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: *** uWSGI is running in multiple interpreter mode *** Jun 21 15:39:07.417010 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: spawned uWSGI master process (pid: 86647) Jun 21 15:39:07.417267 dv-cinder-840678-13-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 21 15:39:07.421805 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: spawned uWSGI worker 1 (pid: 86657, cores: 1) Jun 21 15:39:07.421805 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: spawned uWSGI worker 2 (pid: 86658, cores: 1) Jun 21 15:39:07.421805 dv-cinder-840678-13-iscsi devstack@keystone.service[86647]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 21 15:39:12.879599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 15:39:12.879599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: from cryptography.utils import int_from_bytes Jun 21 15:39:12.880447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 15:39:12.880447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: from cryptography.utils import int_from_bytes Jun 21 15:39:12.902111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 15:39:12.902111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: from cryptography.utils import int_from_bytes Jun 21 15:39:12.902676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 21 15:39:12.902676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: from cryptography.utils import int_from_bytes Jun 21 15:39:12.930396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 15:39:12.930555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 15:39:12.930758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] command line args: [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 15:39:12.931026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 15:39:12.931150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ================================================================================ {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 15:39:12.931434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] admin_token = **** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.931649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] config_dir = [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.931845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.932014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] config_source = [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.932230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] debug = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.932429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] list_limit = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] log_config_append = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] log_dir = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.934907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] log_file = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.935162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.935428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.935626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.935857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.936201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.936358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.936751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.936976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.937209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.937433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.937661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.937915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.938161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.938403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.938674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.939173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.939350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] publish_errors = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.939576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.939802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.939975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.940213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.940442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.940673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] standard_threads = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.940897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.941133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.941305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.941477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] use_journal = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.941652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] use_json = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.941842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] use_stderr = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.942016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] use_syslog = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.942241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.942494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.942728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.942985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.943225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.943478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.943819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.944061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.944294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.944540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.944783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.945033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.945264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.945508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.945739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.946005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.946248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.946479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.946727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.946977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.947223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.947455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.947705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.947962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.948211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.948454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.948705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.949411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 21 15:39:12.949609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 21 15:39:12.949914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] command line args: [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 21 15:39:12.950198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 21 15:39:12.950376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ================================================================================ {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 21 15:39:12.950633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] admin_token = **** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.950880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] config_dir = [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.951068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.951251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] config_source = [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.951432 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] debug = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.951654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.951959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.951959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.952128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.952202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.952274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.952502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.952589 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.952731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] list_limit = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.952911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] log_config_append = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.952985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.953087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.953265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] log_dir = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.953428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.953512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] log_file = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.953659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.953895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.953985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.954162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.954255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.954407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.954483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.954585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.954662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.954762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.954912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.954994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.955169 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.955250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.955409 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.955494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.955653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.955736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.955893 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.955968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.956124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.956210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.956358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.956433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.956591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.956672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.956834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.956922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.957072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.957321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.957395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.957519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] publish_errors = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.957673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.957753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.957862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.957997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.958080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.958160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.958230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.958408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.958490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.958642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.958723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.958876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] standard_threads = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.958953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.959106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.959182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.959346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.959428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.959506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.959576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.959682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] use_journal = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.959797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.959872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] use_json = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.960017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] use_stderr = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.960099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.960193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] use_syslog = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.960267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.960422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 21 15:39:12.960503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.960681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.960769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.960930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.961006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.961159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.961242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.961418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.961500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.961648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.961722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.961931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.962953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.963107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.963184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.963344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.963420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.963602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.963698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.963820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.963895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.964050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.964160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.964367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.964583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.964840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.964840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.965083 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.965199 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.965361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.965464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.965558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.965715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.965816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.965983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.966073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.966225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.966308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.966463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.966547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.966697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.966784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.966923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967273 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.967970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.968135 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.968221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.968366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.968482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.968599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.968678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.968830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.968906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969592 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969747 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.969979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.970981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.971139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.971217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.971387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.971464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.971617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.971693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.971844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] auth.external = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.971920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.972071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.972152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.972307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.972390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.972540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.972618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.972788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.972866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] auth.password = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] auth.token = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.973967 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] credential.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.974967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.975126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.975202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.975351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.975436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.975601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.975677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.975834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.975910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.976065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.976142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.976292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.976368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.976526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.976602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.976793 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.976885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.977025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.977107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.977270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.977345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.977514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.977599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.977750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.977855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] federation.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.978965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.979049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.979204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.979292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.979440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.979517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.979665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.979740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.979928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.980074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.980156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.980350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.980434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.980631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.980761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.980860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.981068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.981159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.981336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.981518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.981606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.981824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.981967 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.982046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.982285 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.982362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.982518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.982725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.982804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.982975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.983058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.983210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.983289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.983440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.983519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.983681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.983758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.983916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.984151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.984238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.984387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.984546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.984643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.984904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.984990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.985147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.985364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.985442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] auth.external = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.985602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.985824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.985915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.986146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.986235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.986472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] auth.password = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.986646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.986881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] auth.token = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.986881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.987262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.987340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.987340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.987468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.987539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.987782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.987871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] credential.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.988993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.989072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.989239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.989326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.989473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.989551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.989713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.989819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.989970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.990144 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.990229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.990310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.990434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.990515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.990717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.990972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.991175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.991255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.991441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.991519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.991678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.991758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.991931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] federation.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.992991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.993145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.993234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.993379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.993456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.993610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.993746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.993937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.994065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.994191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.994321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.994446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.994571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.994703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.994830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.994955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.995080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.995204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.995337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.995473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.995692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.995816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.995940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.995940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.996191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.996323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.996448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.996573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.996697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.996827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.996956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.997909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.998998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.999143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.999247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.999393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.999498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.999661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.999747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.999893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:12.999971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.000123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.000200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.000350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.000435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.000586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.000662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.000818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.000895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.001054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.001140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.001295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.001371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.001535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.001611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.001762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.001919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.002940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.003947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.004106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.004184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.004342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.004420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.004575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.004651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.004809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.004884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.005040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.005116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.005284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.005364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.005524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.005600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.005752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.005848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] resource.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006320 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.006959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.007979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.008131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.008213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.008369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.008447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.008601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] role.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.008678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.008843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] role.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.008921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.009069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.009146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.009305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.009380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.009542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.009621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.009807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.009887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.010994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.011071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.011288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.011480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.011558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.011752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.012995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.013162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.013355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.013433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.013587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.013663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.013865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.013947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.014116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.014206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.014362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.014439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.014596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.014674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.014833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.014957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.015061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.015143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.015295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.015383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.015543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.015629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.015775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.015854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] resource.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.016967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.017053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.017199 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.017275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.017429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.017506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.017657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.017744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.017922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] token.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] role.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] role.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.018956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.019111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.019196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.019346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.019422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.019588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.019666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.019829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.019916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.020090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.020169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.020305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.020382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.020541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.020626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.020771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.020849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.021990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.022080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.022244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.022322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.connection = **** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.022507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.022667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.022746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.022910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.022996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.023162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.023239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.023411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.023489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.023642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.023726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.023872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.023950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.024108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.024186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.024347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.024433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.024580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.025014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.025200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.025435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.025672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.025936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.026189 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.026444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.026693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.026884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.026963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.027350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.027447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.027519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.027752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] token.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.027873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.028121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.028360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.028663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.028740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.028952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.029030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.029193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.029279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.029440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.029518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.029685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.029763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.029959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.030051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.030125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86657) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 15:39:13.030190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.030426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.030705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.030905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.031152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.031381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.031622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.031919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.032183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.connection = **** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.032426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.032673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.032910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.033151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.033387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.033623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.033884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.034121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.034366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.034600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.034938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.035181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.035430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.035672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.035924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.036156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.036445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.036706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 21 15:39:13.036877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86658) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 21 15:39:13.061105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.061326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.061550 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.063005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.063237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.063466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.070052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.072141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.098438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.098719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.101507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.101743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.251006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.253192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.301174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.301410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.301487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.301662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.366376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.366653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.366816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.366928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.366999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.367060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.367233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.367359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.367441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.367512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.367718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.367919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.393391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.393613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.393707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.393863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.393948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.394787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.406679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.406815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.419856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420919 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.420999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.421102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86657) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.421203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.421659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86658) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 21 15:39:13.439018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.439172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.439244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 15:39:13.439328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 15:39:13.439440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.439617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.439742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 15:39:13.439811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 21 15:39:13.439917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.440020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.440216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.440345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.440460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.440551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.440743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.440873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.451892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.452184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.452309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.452461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.452605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.452678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.452754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.452970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.461032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.461176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.464425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.464781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.465013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.465107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.465303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.465395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.465584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.465666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.465881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.465965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.466153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.466234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.466403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.466480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.466666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.466755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.466934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.467012 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.467196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.467273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.467463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.467724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.493957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.494797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.495626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.495911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.496995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.497331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.501352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.501748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.502063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.503233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.503839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.504236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.509469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.509797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.512153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.512426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.514380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.514666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.514928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.515240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.515507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.515769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.516041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.516310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.516913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.517192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.517452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.517727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.518017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.518284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.518548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.518808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.535917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.536193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.538775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.539031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.540948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.544245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.544677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.545081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.548018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.548407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.550207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.553442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.553891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.557073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.557569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.557893 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.558203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.558515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.558804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.559081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.561377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.561674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.561980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.562256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.562533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.562807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.578319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.578575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.578876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.579146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.582649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.582928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.583189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.583462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.587066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.588801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.589074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.591563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.592362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.592654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.592911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.593212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.593337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.593465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.593560 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.596778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.597069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.597367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.597666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.598017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.607157 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.607433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.607720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.607998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.609644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.609968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.610272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.611786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.612057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.612336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.612609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.614275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.614570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.614845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.628528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.631898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.633353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.635245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.637011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.637216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.638697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.638963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.639260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.640682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.642497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.644136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.644408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.644679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.646759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.648381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.652049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.652318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.652797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.652901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.653103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.654575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.658285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.658573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.658961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.659230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.662312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.662598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.662878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.668841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.669122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.669396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.670983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86658) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.671275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.671544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.671815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.672079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.672451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.672731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.673016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.673295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.673575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.673872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.674139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.674415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.674686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86658) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.677497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86657) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 21 15:39:13.677787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.678065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.678334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.678600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.678968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.679249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.679513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.679777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.680057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.680325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.680582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.680843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.681111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86657) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 21 15:39:13.712719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f698be59c0 pid: 86658 (default app) Jun 21 15:39:13.716570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a4843-f123-4dca-95e0-5d29d88d9bfe None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:13.716873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a4843-f123-4dca-95e0-5d29d88d9bfe None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:13.717153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a4843-f123-4dca-95e0-5d29d88d9bfe None None] PATH_INFO: `/v3` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:13.720415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f698be59c0 pid: 86657 (default app) Jun 21 15:39:13.721499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Tue Jun 21 15:39:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:21.692152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b61ea4-b658-4344-99ad-e3f566f90939 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:21.693014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b61ea4-b658-4344-99ad-e3f566f90939 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:21.693014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b61ea4-b658-4344-99ad-e3f566f90939 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:21.700193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:21 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:39:21.708488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:21.708940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:21.709030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:21.815216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: INFO dbcounter [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] Registered counter for database keystone Jun 21 15:39:21.847247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG oslo_db.sqlalchemy.engines [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86658) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 15:39:21.855022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writer thread running {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 15:39:22.055688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG passlib.handlers.bcrypt [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86658) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 15:39:22.056094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG passlib.handlers.bcrypt [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86658) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 15:39:22.112890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:22.114646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:22.250559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 21 15:39:22.390794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b61ea4-b658-4344-99ad-e3f566f90939 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:22.463085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: INFO dbcounter [None req-85b61ea4-b658-4344-99ad-e3f566f90939 None None] Registered counter for database keystone Jun 21 15:39:22.484522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG oslo_db.sqlalchemy.engines [None req-85b61ea4-b658-4344-99ad-e3f566f90939 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86657) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 21 15:39:22.492839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writer thread running {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 21 15:39:22.613648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b61ea4-b658-4344-99ad-e3f566f90939 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:22.619217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e829b6e-7365-49cb-8ca7-1f9be7d2ba95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:22.619878 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e829b6e-7365-49cb-8ca7-1f9be7d2ba95 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:22.620158 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e829b6e-7365-49cb-8ca7-1f9be7d2ba95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:22.620426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e829b6e-7365-49cb-8ca7-1f9be7d2ba95 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:22.629025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e829b6e-7365-49cb-8ca7-1f9be7d2ba95 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:22.682321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5e829b6e-7365-49cb-8ca7-1f9be7d2ba95 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 15:39:22.682588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e829b6e-7365-49cb-8ca7-1f9be7d2ba95 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:22.693509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Jun 21 15:39:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:39:22.699843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:22.715198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e04f7-13b6-4011-a269-576e5592b00c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:22.721199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3b753-aa03-44ef-a07d-ed0945dc23de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:22.722360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b753-aa03-44ef-a07d-ed0945dc23de admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:22.722644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b753-aa03-44ef-a07d-ed0945dc23de admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:22.722916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3b753-aa03-44ef-a07d-ed0945dc23de admin admin] PATH_INFO: `/v3/projects/eb46a2622cdd481281f00889cb2ab74f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:22.727136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3b753-aa03-44ef-a07d-ed0945dc23de admin admin] RBAC: Authorizing `identity:get_project(project_id=eb46a2622cdd481281f00889cb2ab74f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:22.779565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c7f3b753-aa03-44ef-a07d-ed0945dc23de admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 21 15:39:22.779899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3b753-aa03-44ef-a07d-ed0945dc23de admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:22.784730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Tue Jun 21 15:39:22 2022] GET /identity/v3/projects/eb46a2622cdd481281f00889cb2ab74f => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:24.846857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2c4e2-046d-4e6b-8470-971cfdec9fde None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:24.847428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2c4e2-046d-4e6b-8470-971cfdec9fde None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:24.847428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2c4e2-046d-4e6b-8470-971cfdec9fde None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:24.851571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:24.857800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc262e-3e07-4474-9632-fb4eed73f486 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:24.858310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc262e-3e07-4474-9632-fb4eed73f486 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:24.858404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc262e-3e07-4474-9632-fb4eed73f486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:24.880140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b4cc262e-3e07-4474-9632-fb4eed73f486 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:24.881728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b4cc262e-3e07-4474-9632-fb4eed73f486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:24.890810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 21 15:39:24.914774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2c4e2-046d-4e6b-8470-971cfdec9fde None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:24.931451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2c4e2-046d-4e6b-8470-971cfdec9fde None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:24.940302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6a751-023c-436f-9cda-ce6e3f97aa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:24.941404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6a751-023c-436f-9cda-ce6e3f97aa0e admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:24.941942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6a751-023c-436f-9cda-ce6e3f97aa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:24.942432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6a751-023c-436f-9cda-ce6e3f97aa0e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:24.957210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6a751-023c-436f-9cda-ce6e3f97aa0e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:24.960307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6a751-023c-436f-9cda-ce6e3f97aa0e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:24.975332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Jun 21 15:39:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 15:39:24.982260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc262e-3e07-4474-9632-fb4eed73f486 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:24.992557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc262e-3e07-4474-9632-fb4eed73f486 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:24.997209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede97742-aff6-4922-a9a7-b23e5eee523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:24.998141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede97742-aff6-4922-a9a7-b23e5eee523b admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:24.998426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede97742-aff6-4922-a9a7-b23e5eee523b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:24.998696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede97742-aff6-4922-a9a7-b23e5eee523b admin admin] PATH_INFO: `/v3/users/945ae273f8ef413cad49cd7331140c10` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:25.000143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede97742-aff6-4922-a9a7-b23e5eee523b admin admin] RBAC: Authorizing `identity:get_user(user_id=945ae273f8ef413cad49cd7331140c10)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:25.002053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede97742-aff6-4922-a9a7-b23e5eee523b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:25.003698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Tue Jun 21 15:39:24 2022] GET /identity/v3/users/945ae273f8ef413cad49cd7331140c10 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:27.076352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40fd6f-48a4-449d-a39b-71fb190c3f01 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.076906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40fd6f-48a4-449d-a39b-71fb190c3f01 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.076906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40fd6f-48a4-449d-a39b-71fb190c3f01 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.078309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:27.083639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479be88-b856-41a2-941b-9e39fb3330dd None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.084032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479be88-b856-41a2-941b-9e39fb3330dd None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.084229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479be88-b856-41a2-941b-9e39fb3330dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.102118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-4479be88-b856-41a2-941b-9e39fb3330dd None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:27.103293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4479be88-b856-41a2-941b-9e39fb3330dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:27.174902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2280af3-1ecb-4c04-8481-4443d11bc0e8 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.175192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2280af3-1ecb-4c04-8481-4443d11bc0e8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.175470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2280af3-1ecb-4c04-8481-4443d11bc0e8 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.176849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:27.181811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.182072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.182350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.194543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:27.218367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479be88-b856-41a2-941b-9e39fb3330dd None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:27.223507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG passlib.handlers.bcrypt [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86657) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 21 15:39:27.223943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG passlib.handlers.bcrypt [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86657) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 21 15:39:27.243027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:27.269938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:27.271556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:27.330406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479be88-b856-41a2-941b-9e39fb3330dd None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:27.364405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258e06-445f-462c-b026-2a8867766378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:27.365217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258e06-445f-462c-b026-2a8867766378 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.365503 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258e06-445f-462c-b026-2a8867766378 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.365804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39258e06-445f-462c-b026-2a8867766378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.374706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:27.377263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39258e06-445f-462c-b026-2a8867766378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-39258e06-445f-462c-b026-2a8867766378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:27.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:27.389547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:27.390484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:27.395849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:27.397053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258e06-445f-462c-b026-2a8867766378 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:27.479532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:27.497930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e4e3a-4189-42fe-b31c-182386bd2ed2 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:27.499043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39258e06-445f-462c-b026-2a8867766378 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:27.500314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:27.534895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6982c-1f9f-4462-934c-7bd37e1a0cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:27.535557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6982c-1f9f-4462-934c-7bd37e1a0cbc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.535837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6982c-1f9f-4462-934c-7bd37e1a0cbc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.536116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6982c-1f9f-4462-934c-7bd37e1a0cbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.536467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:27.537395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.537679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.537987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.538951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:27.539245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:27.540689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:27.553075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:27.558774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:27.605375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6982c-1f9f-4462-934c-7bd37e1a0cbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:27.606916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6982c-1f9f-4462-934c-7bd37e1a0cbc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:27.662936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a05d3f-b6d9-4a67-9414-58dc176aa746 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:27.689410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:27.698537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb481ee8-33eb-4436-880e-c4da29e76222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:27.700175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb481ee8-33eb-4436-880e-c4da29e76222 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.700466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb481ee8-33eb-4436-880e-c4da29e76222 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.700745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb481ee8-33eb-4436-880e-c4da29e76222 None admin] PATH_INFO: `/v3/users/945ae273f8ef413cad49cd7331140c10` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.702376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb481ee8-33eb-4436-880e-c4da29e76222 None admin] RBAC: Authorizing `identity:get_user(user_id=945ae273f8ef413cad49cd7331140c10)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:27.703833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb481ee8-33eb-4436-880e-c4da29e76222 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:27.705705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 9/16] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:27 2022] GET /identity/v3/users/945ae273f8ef413cad49cd7331140c10 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:27.709509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 8/17] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 15:39:27.710847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb481ee8-33eb-4436-880e-c4da29e76222 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:27.715058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6982c-1f9f-4462-934c-7bd37e1a0cbc None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:27.745754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:27.816130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6982c-1f9f-4462-934c-7bd37e1a0cbc None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:27.817111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb481ee8-33eb-4436-880e-c4da29e76222 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:27.852069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f974c8-2065-4fce-9c57-7f4fd3c05f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:27.852378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:27.852872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f974c8-2065-4fce-9c57-7f4fd3c05f27 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.853157 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f974c8-2065-4fce-9c57-7f4fd3c05f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.853241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:27.853433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f974c8-2065-4fce-9c57-7f4fd3c05f27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.853516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:27.853721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:27.854684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f974c8-2065-4fce-9c57-7f4fd3c05f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:27.856283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f974c8-2065-4fce-9c57-7f4fd3c05f27 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:27.857938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:27.864143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f974c8-2065-4fce-9c57-7f4fd3c05f27 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:27.864950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:27.866912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:39:27.888371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:27.889272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:27.890121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:39:27.890121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:39:27.890121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:39:27.890121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:27.890121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:39:27.895383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:27.933105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:27.970377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f974c8-2065-4fce-9c57-7f4fd3c05f27 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:27.971980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:28.006331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fc782-a296-4185-a54b-3f4817ca37b1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:28.014181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb32c8-cefa-47d9-bfbc-4b12e3d6e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:28.014862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb32c8-cefa-47d9-bfbc-4b12e3d6e497 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:28.015148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb32c8-cefa-47d9-bfbc-4b12e3d6e497 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:28.015433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb32c8-cefa-47d9-bfbc-4b12e3d6e497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:28.016268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb32c8-cefa-47d9-bfbc-4b12e3d6e497 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.domain.id=default, user.id=945ae273f8ef413cad49cd7331140c10) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:28.016583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb32c8-cefa-47d9-bfbc-4b12e3d6e497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:28.018082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb32c8-cefa-47d9-bfbc-4b12e3d6e497 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:28.027049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Jun 21 15:39:27 2022] GET /identity/v3/role_assignments?user.id=945ae273f8ef413cad49cd7331140c10&scope.domain.id=default&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 233 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:28.041170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebba7a4-9fed-421d-9732-a05da01074dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:28.041965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba7a4-9fed-421d-9732-a05da01074dc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:28.042250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba7a4-9fed-421d-9732-a05da01074dc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:28.042523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba7a4-9fed-421d-9732-a05da01074dc None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:28.043437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebba7a4-9fed-421d-9732-a05da01074dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:28.043755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebba7a4-9fed-421d-9732-a05da01074dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:28.045240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebba7a4-9fed-421d-9732-a05da01074dc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:28.064725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:30.458566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f788a5-4f32-43ed-a468-9391c01a6b8d None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:30.459200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f788a5-4f32-43ed-a468-9391c01a6b8d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:30.459200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f788a5-4f32-43ed-a468-9391c01a6b8d None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:30.460533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:30.466056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eaa7bc-cd72-4086-8b1c-e5e6d6be0656 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:30.466926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eaa7bc-cd72-4086-8b1c-e5e6d6be0656 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:30.467435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eaa7bc-cd72-4086-8b1c-e5e6d6be0656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:30.490817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-63eaa7bc-cd72-4086-8b1c-e5e6d6be0656 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:30.492375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-63eaa7bc-cd72-4086-8b1c-e5e6d6be0656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:30.527303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678bc2b-803a-412b-8721-f1b3bb0183e3 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:30.528016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678bc2b-803a-412b-8721-f1b3bb0183e3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:30.528512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678bc2b-803a-412b-8721-f1b3bb0183e3 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:30.530136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 13/23] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:39:30.538066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6b4e0-5cd6-41c7-bc4f-a019b5fd7f9b None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:30.538648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6b4e0-5cd6-41c7-bc4f-a019b5fd7f9b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:30.539138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6b4e0-5cd6-41c7-bc4f-a019b5fd7f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:30.570799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0eb6b4e0-5cd6-41c7-bc4f-a019b5fd7f9b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:30.572497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0eb6b4e0-5cd6-41c7-bc4f-a019b5fd7f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:30.636526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 11/24] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:30.667439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eaa7bc-cd72-4086-8b1c-e5e6d6be0656 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:30.718979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:30.755191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:39:30.790975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6b4e0-5cd6-41c7-bc4f-a019b5fd7f9b None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:30.851984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eaa7bc-cd72-4086-8b1c-e5e6d6be0656 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:30.911284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:30.920276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6b4e0-5cd6-41c7-bc4f-a019b5fd7f9b None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:30.932342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee940b57-57cb-4e47-a469-c9ac6f52d1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:30.933461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee940b57-57cb-4e47-a469-c9ac6f52d1f7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:30.934184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee940b57-57cb-4e47-a469-c9ac6f52d1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:30.934699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee940b57-57cb-4e47-a469-c9ac6f52d1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:30.952230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee940b57-57cb-4e47-a469-c9ac6f52d1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:30.955343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee940b57-57cb-4e47-a469-c9ac6f52d1f7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:30.985282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3662ab1-d643-4b1f-b74c-8fabe3beef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:30.986508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662ab1-d643-4b1f-b74c-8fabe3beef28 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:30.986988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662ab1-d643-4b1f-b74c-8fabe3beef28 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:30.987549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662ab1-d643-4b1f-b74c-8fabe3beef28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-ee940b57-57cb-4e47-a469-c9ac6f52d1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:39:30.990408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:30.991201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:30.991975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:39:30.991975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:39:30.991975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:39:30.991975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:30.995576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 12/26] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:39:30.999633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88683f29-2d08-4e3f-a0a6-f726271d25c2 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.001447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88683f29-2d08-4e3f-a0a6-f726271d25c2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.002147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88683f29-2d08-4e3f-a0a6-f726271d25c2 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.011875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 13/27] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:31.013471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88683f29-2d08-4e3f-a0a6-f726271d25c2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:31.014964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3662ab1-d643-4b1f-b74c-8fabe3beef28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-a3662ab1-d643-4b1f-b74c-8fabe3beef28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:31.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:31.037833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:31.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:31.043799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 15/28] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:39:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:31.044842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b792b6e-7f59-4cf0-98fc-a9ba2673ec94 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.045135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b792b6e-7f59-4cf0-98fc-a9ba2673ec94 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.045413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b792b6e-7f59-4cf0-98fc-a9ba2673ec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.076202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0b792b6e-7f59-4cf0-98fc-a9ba2673ec94 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:31.077443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0b792b6e-7f59-4cf0-98fc-a9ba2673ec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:31.152689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:31.184221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 16/29] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:31.185320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b792b6e-7f59-4cf0-98fc-a9ba2673ec94 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:31.186182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88683f29-2d08-4e3f-a0a6-f726271d25c2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:31.232292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd9706-b6d0-4690-bd4d-b48453c22d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:31.233279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd9706-b6d0-4690-bd4d-b48453c22d58 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.233697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd9706-b6d0-4690-bd4d-b48453c22d58 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.234237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd9706-b6d0-4690-bd4d-b48453c22d58 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.235184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd9706-b6d0-4690-bd4d-b48453c22d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:31.235624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd9706-b6d0-4690-bd4d-b48453c22d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:31.237367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd9706-b6d0-4690-bd4d-b48453c22d58 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:31.253850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 14/30] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:31.256223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f0e7-7c72-4bd0-9926-4597321b6660 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.256947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f0e7-7c72-4bd0-9926-4597321b6660 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.257420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f0e7-7c72-4bd0-9926-4597321b6660 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.259235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 15/31] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:31.260107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469f0e7-7c72-4bd0-9926-4597321b6660 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:31.307350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:31.325890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b792b6e-7f59-4cf0-98fc-a9ba2673ec94 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:31.366863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:31.367548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.367831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.368114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.368931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:31.369364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:31.370890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:31.374481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469f0e7-7c72-4bd0-9926-4597321b6660 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:31.381238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 17/32] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:31.382072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:31.411338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd075d-9c3c-461b-bc83-08e3a13c8082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:31.412375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd075d-9c3c-461b-bc83-08e3a13c8082 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.412853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd075d-9c3c-461b-bc83-08e3a13c8082 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.413345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd075d-9c3c-461b-bc83-08e3a13c8082 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.414347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd075d-9c3c-461b-bc83-08e3a13c8082 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:31.415972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd075d-9c3c-461b-bc83-08e3a13c8082 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:31.421638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:31.495849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f618-29d1-41f4-81b0-85da34b08a17 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:31.531638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49718db6-8ca8-4fc7-bd72-c7404166184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:31.532680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49718db6-8ca8-4fc7-bd72-c7404166184f None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.533101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49718db6-8ca8-4fc7-bd72-c7404166184f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.533506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49718db6-8ca8-4fc7-bd72-c7404166184f None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.534375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49718db6-8ca8-4fc7-bd72-c7404166184f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:31.535980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49718db6-8ca8-4fc7-bd72-c7404166184f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:31.548722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 16/33] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jun 21 15:39:31 2022] POST /identity/v3/roles => generated 225 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:31.550942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac5c6f-a355-4dfc-85e1-5f55778e7d28 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.551443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac5c6f-a355-4dfc-85e1-5f55778e7d28 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.551920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac5c6f-a355-4dfc-85e1-5f55778e7d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.572803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-34ac5c6f-a355-4dfc-85e1-5f55778e7d28 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:31.574921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-34ac5c6f-a355-4dfc-85e1-5f55778e7d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:31.591947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:31.656717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 17/34] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:31.657926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac5c6f-a355-4dfc-85e1-5f55778e7d28 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:31.776093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 18/35] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:39:31 2022] POST /identity/v3/projects => generated 301 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:31.777605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49718db6-8ca8-4fc7-bd72-c7404166184f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:31.778674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac5c6f-a355-4dfc-85e1-5f55778e7d28 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:31.828992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:31.858539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd9900-9893-40ed-9cef-eba8ef469d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:31.859598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd9900-9893-40ed-9cef-eba8ef469d19 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.859886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd9900-9893-40ed-9cef-eba8ef469d19 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.860166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd9900-9893-40ed-9cef-eba8ef469d19 None admin] PATH_INFO: `/v3/users/945ae273f8ef413cad49cd7331140c10` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.861481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd9900-9893-40ed-9cef-eba8ef469d19 None admin] RBAC: Authorizing `identity:get_user(user_id=945ae273f8ef413cad49cd7331140c10)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:31.863384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd9900-9893-40ed-9cef-eba8ef469d19 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:31.865855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 18/36] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:31 2022] GET /identity/v3/users/945ae273f8ef413cad49cd7331140c10 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:31.867462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d4e3c-62be-42b6-b5a0-3eb596227409 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.867928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d4e3c-62be-42b6-b5a0-3eb596227409 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.868348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d4e3c-62be-42b6-b5a0-3eb596227409 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.870661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 19/37] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:31.872121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d4e3c-62be-42b6-b5a0-3eb596227409 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:31.901935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49718db6-8ca8-4fc7-bd72-c7404166184f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:31.946447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49df8b4-8526-4e6a-8b21-967dbf73d380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:31.947690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df8b4-8526-4e6a-8b21-967dbf73d380 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.948220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df8b4-8526-4e6a-8b21-967dbf73d380 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.948716 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df8b4-8526-4e6a-8b21-967dbf73d380 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:31.950235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49df8b4-8526-4e6a-8b21-967dbf73d380 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:31.952161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49df8b4-8526-4e6a-8b21-967dbf73d380 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:31.974079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 19/38] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jun 21 15:39:31 2022] POST /identity/v3/roles => generated 219 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:31.975955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622bdf3-d7c4-4389-988a-2b0a101ae5d1 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:31.976436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622bdf3-d7c4-4389-988a-2b0a101ae5d1 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:31.977012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622bdf3-d7c4-4389-988a-2b0a101ae5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:32.003071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7622bdf3-d7c4-4389-988a-2b0a101ae5d1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:32.005919 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7622bdf3-d7c4-4389-988a-2b0a101ae5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:32.017947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d4e3c-62be-42b6-b5a0-3eb596227409 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:32.064941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f8845-1e9e-4dbe-9852-9a0774b397c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:32.065884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f8845-1e9e-4dbe-9852-9a0774b397c3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:32.066180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f8845-1e9e-4dbe-9852-9a0774b397c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:32.066459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f8845-1e9e-4dbe-9852-9a0774b397c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:32.068405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f8845-1e9e-4dbe-9852-9a0774b397c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:32.070029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f8845-1e9e-4dbe-9852-9a0774b397c3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:32.071781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 20/39] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:32.077742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f8845-1e9e-4dbe-9852-9a0774b397c3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:32.102952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 20/40] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:32.103512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:32.134209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622bdf3-d7c4-4389-988a-2b0a101ae5d1 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:32.168412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:32.215514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f8845-1e9e-4dbe-9852-9a0774b397c3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:32.265617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622bdf3-d7c4-4389-988a-2b0a101ae5d1 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:32.302574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e49f0-701a-4007-87b7-3b7f336b2334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:32.303401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e49f0-701a-4007-87b7-3b7f336b2334 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:32.303688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e49f0-701a-4007-87b7-3b7f336b2334 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:32.303965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e49f0-701a-4007-87b7-3b7f336b2334 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:32.305276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e49f0-701a-4007-87b7-3b7f336b2334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:32.306826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e49f0-701a-4007-87b7-3b7f336b2334 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:32.308687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 21/41] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:32.310141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d508d9-955e-45c1-b10f-1805af1e55d5 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:32.310434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d508d9-955e-45c1-b10f-1805af1e55d5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:32.310710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d508d9-955e-45c1-b10f-1805af1e55d5 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:32.312250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 22/42] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:32.315601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d508d9-955e-45c1-b10f-1805af1e55d5 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:32.368051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd8a37-080b-4d96-980c-435293414bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:32.368855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8a37-080b-4d96-980c-435293414bbf None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:32.369137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8a37-080b-4d96-980c-435293414bbf None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:32.369414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd8a37-080b-4d96-980c-435293414bbf None admin] PATH_INFO: `/v3/domains/default/users/945ae273f8ef413cad49cd7331140c10/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:32.372281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd8a37-080b-4d96-980c-435293414bbf None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=945ae273f8ef413cad49cd7331140c10, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:32.373759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd8a37-080b-4d96-980c-435293414bbf None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:32.385571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:32.389346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 21/43] 10.222.0.52 () {58 vars in 1361 bytes} [Tue Jun 21 15:39:32 2022] PUT /identity/v3/domains/default/users/945ae273f8ef413cad49cd7331140c10/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:39:32.390753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496590a9-cfbe-48f8-aa8c-ae8a35f73ddd None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:32.391049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496590a9-cfbe-48f8-aa8c-ae8a35f73ddd None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:32.391335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496590a9-cfbe-48f8-aa8c-ae8a35f73ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:32.409599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-496590a9-cfbe-48f8-aa8c-ae8a35f73ddd None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:32.410916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-496590a9-cfbe-48f8-aa8c-ae8a35f73ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:32.445965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d508d9-955e-45c1-b10f-1805af1e55d5 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:32.487266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 22/44] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:32.489087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f7633-8507-4d7d-9682-fa3bc73b839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:32.489999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7633-8507-4d7d-9682-fa3bc73b839e None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:32.490420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7633-8507-4d7d-9682-fa3bc73b839e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:32.490828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f7633-8507-4d7d-9682-fa3bc73b839e None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:32.491712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f7633-8507-4d7d-9682-fa3bc73b839e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:32.493340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f7633-8507-4d7d-9682-fa3bc73b839e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:32.510242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496590a9-cfbe-48f8-aa8c-ae8a35f73ddd None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:32.526207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 23/45] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:39:32 2022] POST /identity/v3/projects => generated 312 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:32.602186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:32.658938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496590a9-cfbe-48f8-aa8c-ae8a35f73ddd None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:32.702264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f51b0-fa91-4cae-a8da-9d95c48fb1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:32.702973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f51b0-fa91-4cae-a8da-9d95c48fb1d9 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:32.703271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f51b0-fa91-4cae-a8da-9d95c48fb1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:32.703533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f51b0-fa91-4cae-a8da-9d95c48fb1d9 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:32.704318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f51b0-fa91-4cae-a8da-9d95c48fb1d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:32.705726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f51b0-fa91-4cae-a8da-9d95c48fb1d9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:32.721464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jun 21 15:39:32 2022] POST /identity/v3/roles => generated 223 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:39:34.446903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf8f3a-7620-4370-96da-e83a6c4d782e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:34.447456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf8f3a-7620-4370-96da-e83a6c4d782e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:34.447456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf8f3a-7620-4370-96da-e83a6c4d782e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:34.448873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 24/47] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:34.454320 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16952938-8ca8-42af-8043-a00db2332b89 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:34.454745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16952938-8ca8-42af-8043-a00db2332b89 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:34.454969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16952938-8ca8-42af-8043-a00db2332b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:34.473642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-16952938-8ca8-42af-8043-a00db2332b89 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:34.474893 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-16952938-8ca8-42af-8043-a00db2332b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:34.544187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 24/48] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:34.567985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf8f3a-7620-4370-96da-e83a6c4d782e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:34.640170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4094a-b676-4d43-b70c-862d794fa6b6 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:34.640437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4094a-b676-4d43-b70c-862d794fa6b6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:34.640717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4094a-b676-4d43-b70c-862d794fa6b6 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:34.642247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 25/49] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:39:34.647264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2f98-9f3b-498a-bcd9-d3c1136d456b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:34.647548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2f98-9f3b-498a-bcd9-d3c1136d456b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:34.647828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d2f98-9f3b-498a-bcd9-d3c1136d456b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:34.665710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c02d2f98-9f3b-498a-bcd9-d3c1136d456b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:34.666934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c02d2f98-9f3b-498a-bcd9-d3c1136d456b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:34.676858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf8f3a-7620-4370-96da-e83a6c4d782e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:34.688976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:34.704475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:34.715391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf435e1-e8ad-4035-b4ef-0cee55b19808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:34.716035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf435e1-e8ad-4035-b4ef-0cee55b19808 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:34.716327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf435e1-e8ad-4035-b4ef-0cee55b19808 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:34.716607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf435e1-e8ad-4035-b4ef-0cee55b19808 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:34.727994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf435e1-e8ad-4035-b4ef-0cee55b19808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-dcf435e1-e8ad-4035-b4ef-0cee55b19808 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:34.735869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:34.736734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:34.737684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 25/50] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:39:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:34.738593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 26/51] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:39:34.743080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf435e1-e8ad-4035-b4ef-0cee55b19808 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:34.761500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d2f98-9f3b-498a-bcd9-d3c1136d456b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:34.847247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf435e1-e8ad-4035-b4ef-0cee55b19808 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:34.864394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d2f98-9f3b-498a-bcd9-d3c1136d456b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:34.882500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:34.883179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:34.883461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:34.883737 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:34.884721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:34.884934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:34.887678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:34.898492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 26/52] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:34.904495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:34.906722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828a6c2-6810-4d79-9d01-bd5ec0439c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:34.908088 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828a6c2-6810-4d79-9d01-bd5ec0439c68 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:34.908436 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828a6c2-6810-4d79-9d01-bd5ec0439c68 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:34.909004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828a6c2-6810-4d79-9d01-bd5ec0439c68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:34.911251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828a6c2-6810-4d79-9d01-bd5ec0439c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:34.913847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828a6c2-6810-4d79-9d01-bd5ec0439c68 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:34.916705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:34.924624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828a6c2-6810-4d79-9d01-bd5ec0439c68 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:34.944176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:34.966589 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:35.022902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828a6c2-6810-4d79-9d01-bd5ec0439c68 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:35.038814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a60777-e59c-4ff8-ba09-ad4a76859717 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:35.059017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecea28-6b8d-4569-b1f9-dd6e60795b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:35.059365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecea28-6b8d-4569-b1f9-dd6e60795b34 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:35.059654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecea28-6b8d-4569-b1f9-dd6e60795b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:35.059935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecea28-6b8d-4569-b1f9-dd6e60795b34 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:35.060657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecea28-6b8d-4569-b1f9-dd6e60795b34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:35.062162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecea28-6b8d-4569-b1f9-dd6e60795b34 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:35.089399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 28/54] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Jun 21 15:39:34 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:35.095021 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df35c-61ff-4a17-bef6-c4451b4eba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:35.096354 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df35c-61ff-4a17-bef6-c4451b4eba80 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:35.096856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df35c-61ff-4a17-bef6-c4451b4eba80 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:35.097416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40df35c-61ff-4a17-bef6-c4451b4eba80 None admin] PATH_INFO: `/v3/users/945ae273f8ef413cad49cd7331140c10` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:35.099763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40df35c-61ff-4a17-bef6-c4451b4eba80 None admin] RBAC: Authorizing `identity:get_user(user_id=945ae273f8ef413cad49cd7331140c10)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:35.102608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40df35c-61ff-4a17-bef6-c4451b4eba80 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:35.104489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 27/55] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:34 2022] GET /identity/v3/users/945ae273f8ef413cad49cd7331140c10 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:35.110483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecea28-6b8d-4569-b1f9-dd6e60795b34 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:35.213634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:35.224714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecea28-6b8d-4569-b1f9-dd6e60795b34 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:35.265544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f336c3c-44d4-4011-b130-e633645bcded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:35.266429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f336c3c-44d4-4011-b130-e633645bcded None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:35.266716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f336c3c-44d4-4011-b130-e633645bcded None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:35.266992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f336c3c-44d4-4011-b130-e633645bcded None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:35.268197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f336c3c-44d4-4011-b130-e633645bcded None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:35.269673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f336c3c-44d4-4011-b130-e633645bcded None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:35.271348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 29/56] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:35.277678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df35c-61ff-4a17-bef6-c4451b4eba80 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:35.496155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:35.499180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40df35c-61ff-4a17-bef6-c4451b4eba80 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:35.534782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2607f99-e77b-4171-89c3-a18a25ba543c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:35.535457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2607f99-e77b-4171-89c3-a18a25ba543c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:35.535737 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2607f99-e77b-4171-89c3-a18a25ba543c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:35.536002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2607f99-e77b-4171-89c3-a18a25ba543c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:35.536852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2607f99-e77b-4171-89c3-a18a25ba543c None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.domain.id=default, user.id=945ae273f8ef413cad49cd7331140c10) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:35.537154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2607f99-e77b-4171-89c3-a18a25ba543c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:35.538656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2607f99-e77b-4171-89c3-a18a25ba543c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:35.545174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 28/57] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Jun 21 15:39:35 2022] GET /identity/v3/role_assignments?user.id=945ae273f8ef413cad49cd7331140c10&scope.domain.id=default&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 530 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:37.161316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4787ce7-b631-4f52-a72e-c239806e16a3 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:37.162044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4787ce7-b631-4f52-a72e-c239806e16a3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:37.162191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4787ce7-b631-4f52-a72e-c239806e16a3 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:37.163523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 30/58] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:37.168885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfede9-5869-4b13-b4e7-fd8f114da52e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:37.169320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfede9-5869-4b13-b4e7-fd8f114da52e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:37.169404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfede9-5869-4b13-b4e7-fd8f114da52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:37.187295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e7bfede9-5869-4b13-b4e7-fd8f114da52e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:37.188524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e7bfede9-5869-4b13-b4e7-fd8f114da52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:37.260444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 29/59] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:37.285562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4787ce7-b631-4f52-a72e-c239806e16a3 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:37.395046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4787ce7-b631-4f52-a72e-c239806e16a3 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:37.431598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4d875-19e9-4549-a6ec-4a03f37374c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:37.432330 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4d875-19e9-4549-a6ec-4a03f37374c1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:37.432610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4d875-19e9-4549-a6ec-4a03f37374c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:37.432888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4d875-19e9-4549-a6ec-4a03f37374c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:37.434631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4d875-19e9-4549-a6ec-4a03f37374c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:37.436988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4d875-19e9-4549-a6ec-4a03f37374c1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:37.438736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 31/60] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:37.447801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfede9-5869-4b13-b4e7-fd8f114da52e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:37.549208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:37.557431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfede9-5869-4b13-b4e7-fd8f114da52e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:37.592838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8233182-5e0c-48e2-a2b7-0004504b76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:37.593526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8233182-5e0c-48e2-a2b7-0004504b76b9 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:37.593837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8233182-5e0c-48e2-a2b7-0004504b76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:37.594121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8233182-5e0c-48e2-a2b7-0004504b76b9 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:37.594865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8233182-5e0c-48e2-a2b7-0004504b76b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:37.596320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8233182-5e0c-48e2-a2b7-0004504b76b9 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:37.622825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 30/61] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:39:37 2022] POST /identity/v3/users => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:40.981006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce391b-06ac-436a-9375-e1dcd7ac39df None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:40.982131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce391b-06ac-436a-9375-e1dcd7ac39df None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:40.982639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce391b-06ac-436a-9375-e1dcd7ac39df None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:40.984539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 32/62] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:40.989810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2844d2-850b-4a72-bdee-cecff41dc682 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:40.990658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2844d2-850b-4a72-bdee-cecff41dc682 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:40.991176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2844d2-850b-4a72-bdee-cecff41dc682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.014838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7d2844d2-850b-4a72-bdee-cecff41dc682 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:41.016376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7d2844d2-850b-4a72-bdee-cecff41dc682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:41.112524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 31/63] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:41.159290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce391b-06ac-436a-9375-e1dcd7ac39df None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:41.211235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:41.252188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b36676-cd54-4d94-9935-14b7b795acd6 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.252730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b36676-cd54-4d94-9935-14b7b795acd6 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.253203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b36676-cd54-4d94-9935-14b7b795acd6 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.255080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 32/64] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:41.257051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b66f7e-77b7-48f2-aea7-13d7d0456211 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.257726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b66f7e-77b7-48f2-aea7-13d7d0456211 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.258319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b66f7e-77b7-48f2-aea7-13d7d0456211 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.260072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 33/65] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:41.266767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6dd0a-e361-46d3-9020-e381bf35f3d5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.267371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6dd0a-e361-46d3-9020-e381bf35f3d5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.267757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6dd0a-e361-46d3-9020-e381bf35f3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.291330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e1b6dd0a-e361-46d3-9020-e381bf35f3d5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:41.292528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e1b6dd0a-e361-46d3-9020-e381bf35f3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:41.322009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce391b-06ac-436a-9375-e1dcd7ac39df None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:41.357267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b08f2e-f9ef-4117-aa35-031be30a717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:41.358095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b08f2e-f9ef-4117-aa35-031be30a717b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.358384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b08f2e-f9ef-4117-aa35-031be30a717b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.358686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b08f2e-f9ef-4117-aa35-031be30a717b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.374626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 34/66] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:41.374626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709a923-0acd-4c0a-b06d-d36e99e03a80 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.374626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709a923-0acd-4c0a-b06d-d36e99e03a80 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.374626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709a923-0acd-4c0a-b06d-d36e99e03a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.378810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b08f2e-f9ef-4117-aa35-031be30a717b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-23b08f2e-f9ef-4117-aa35-031be30a717b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:41.393464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:41.394427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:41.395175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:41.395767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8709a923-0acd-4c0a-b06d-d36e99e03a80 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:41.395845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 33/67] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:41.396990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8709a923-0acd-4c0a-b06d-d36e99e03a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:41.397930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b941f3-b75e-4501-a496-cda2dc62fc2d None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.398712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b941f3-b75e-4501-a496-cda2dc62fc2d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.399782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b941f3-b75e-4501-a496-cda2dc62fc2d None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.402608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:41.403760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 34/68] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:41.405331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b941f3-b75e-4501-a496-cda2dc62fc2d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:41.478736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 35/69] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:41.486830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709a923-0acd-4c0a-b06d-d36e99e03a80 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:41.522546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:41.539181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b941f3-b75e-4501-a496-cda2dc62fc2d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:41.581664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe8acb1-5a90-4d07-ab17-b8aec8ff0b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:41.583228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8acb1-5a90-4d07-ab17-b8aec8ff0b13 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.583228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8acb1-5a90-4d07-ab17-b8aec8ff0b13 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.583228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8acb1-5a90-4d07-ab17-b8aec8ff0b13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.595032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe8acb1-5a90-4d07-ab17-b8aec8ff0b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-fbe8acb1-5a90-4d07-ab17-b8aec8ff0b13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:41.603204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:41.604302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:41.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 35/70] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:39:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:41.606332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96004ade-1450-48cd-a2e5-0b02687f6c7b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.606332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96004ade-1450-48cd-a2e5-0b02687f6c7b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.606600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96004ade-1450-48cd-a2e5-0b02687f6c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.615489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709a923-0acd-4c0a-b06d-d36e99e03a80 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:41.624579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-96004ade-1450-48cd-a2e5-0b02687f6c7b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:41.624579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-96004ade-1450-48cd-a2e5-0b02687f6c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:41.654833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:41.655634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.655927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.656241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.657009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] RBAC: Adding query filter params (name=member) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:41.657353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:41.658786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:41.668879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 36/71] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:41.669750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:41.673023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:41.700906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 36/72] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:41.702511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3f000-adbd-4619-bbaf-fe649bc62b2d None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.702691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3f000-adbd-4619-bbaf-fe649bc62b2d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.702975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3f000-adbd-4619-bbaf-fe649bc62b2d None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.704288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 37/73] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:41.705121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3f000-adbd-4619-bbaf-fe649bc62b2d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:41.786055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1c7e-0222-43af-9b5e-edd0d7585568 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:41.797099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:41.822059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3f000-adbd-4619-bbaf-fe649bc62b2d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:41.823071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb395b-a037-4de7-b0ff-32646b7e1fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:41.824381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb395b-a037-4de7-b0ff-32646b7e1fb6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.824675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb395b-a037-4de7-b0ff-32646b7e1fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.824949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb395b-a037-4de7-b0ff-32646b7e1fb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.836724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb395b-a037-4de7-b0ff-32646b7e1fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-54eb395b-a037-4de7-b0ff-32646b7e1fb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:41.844674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:41.845566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:41.846838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 37/74] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:39:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:41.847444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb395b-a037-4de7-b0ff-32646b7e1fb6 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:41.868393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c15e4-0059-408a-92cf-ad4fbb42c43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:41.869091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c15e4-0059-408a-92cf-ad4fbb42c43f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.869376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c15e4-0059-408a-92cf-ad4fbb42c43f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.869655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c15e4-0059-408a-92cf-ad4fbb42c43f None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.870472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c15e4-0059-408a-92cf-ad4fbb42c43f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:41.870900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c15e4-0059-408a-92cf-ad4fbb42c43f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:41.872454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c15e4-0059-408a-92cf-ad4fbb42c43f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:41.884614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 38/75] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:39:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:41.887529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2b0e4-89cb-4fa4-9bc6-93fc1a7f369d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.887825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2b0e4-89cb-4fa4-9bc6-93fc1a7f369d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.888365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2b0e4-89cb-4fa4-9bc6-93fc1a7f369d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:41.907034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-bbf2b0e4-89cb-4fa4-9bc6-93fc1a7f369d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:41.908256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bbf2b0e4-89cb-4fa4-9bc6-93fc1a7f369d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:41.921307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:41.960865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb395b-a037-4de7-b0ff-32646b7e1fb6 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:41.978765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 39/76] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:41.986685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2b0e4-89cb-4fa4-9bc6-93fc1a7f369d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:41.996598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f61b0-b970-458b-b3ce-26bcbed3fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:41.997511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f61b0-b970-458b-b3ce-26bcbed3fe2d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:41.997825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f61b0-b970-458b-b3ce-26bcbed3fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:41.998112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f61b0-b970-458b-b3ce-26bcbed3fe2d None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.015209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f61b0-b970-458b-b3ce-26bcbed3fe2d None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.016759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f61b0-b970-458b-b3ce-26bcbed3fe2d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.018581 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 38/77] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:41 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:42.019363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f61b0-b970-458b-b3ce-26bcbed3fe2d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.070172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:42.106870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2b0e4-89cb-4fa4-9bc6-93fc1a7f369d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.136692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f61b0-b970-458b-b3ce-26bcbed3fe2d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.158128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248c9e8-34e3-40c4-8669-7287525f18cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.158952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248c9e8-34e3-40c4-8669-7287525f18cb None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.159239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248c9e8-34e3-40c4-8669-7287525f18cb None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.159516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248c9e8-34e3-40c4-8669-7287525f18cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.162130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:42.173392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248c9e8-34e3-40c4-8669-7287525f18cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:42.179261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.179945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.180227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.180506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.181264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:42.181577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-8248c9e8-34e3-40c4-8669-7287525f18cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:42.182120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:42.183023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:42.189706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:42.190481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.190563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 40/78] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:42.190563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248c9e8-34e3-40c4-8669-7287525f18cb None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.201069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 39/79] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:42.202316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.311456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248c9e8-34e3-40c4-8669-7287525f18cb None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.336918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843090d2-b371-4fad-9c51-05ea2657c51d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.342573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:42.352471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e77d18-99a3-4dbd-b058-90b8d01f40c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.353052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e77d18-99a3-4dbd-b058-90b8d01f40c4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.353647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e77d18-99a3-4dbd-b058-90b8d01f40c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.353647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e77d18-99a3-4dbd-b058-90b8d01f40c4 None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.355435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e77d18-99a3-4dbd-b058-90b8d01f40c4 None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.357075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e77d18-99a3-4dbd-b058-90b8d01f40c4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.360020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 41/80] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:42 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:42.361318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e77d18-99a3-4dbd-b058-90b8d01f40c4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.384541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207d8fe-5e5f-4f84-83e6-daa114351573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.385371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d8fe-5e5f-4f84-83e6-daa114351573 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.385659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d8fe-5e5f-4f84-83e6-daa114351573 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.385983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2207d8fe-5e5f-4f84-83e6-daa114351573 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.387336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207d8fe-5e5f-4f84-83e6-daa114351573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.389050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2207d8fe-5e5f-4f84-83e6-daa114351573 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.390894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 40/81] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:42.395529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207d8fe-5e5f-4f84-83e6-daa114351573 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.450273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:42.470807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e77d18-99a3-4dbd-b058-90b8d01f40c4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.507571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f868b0a-3f0e-488c-b90a-6d3b052805b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.509546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868b0a-3f0e-488c-b90a-6d3b052805b1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.509546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868b0a-3f0e-488c-b90a-6d3b052805b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.509546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868b0a-3f0e-488c-b90a-6d3b052805b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.514435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2207d8fe-5e5f-4f84-83e6-daa114351573 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.557613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.558389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.558657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.558935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.559705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] RBAC: Adding query filter params (name=member) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:42.560014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.561460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.576233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 41/82] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:42.576233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.583846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:42.587800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f868b0a-3f0e-488c-b90a-6d3b052805b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.589477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f868b0a-3f0e-488c-b90a-6d3b052805b1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.688724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0b4c5-ddaf-4480-b751-21729d9de122 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.699424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 42/83] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:42.700967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f868b0a-3f0e-488c-b90a-6d3b052805b1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.703888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:42.727051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c62b2-c9d0-401d-bc87-c429c5efe755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.727948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c62b2-c9d0-401d-bc87-c429c5efe755 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.728223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c62b2-c9d0-401d-bc87-c429c5efe755 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.728508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c62b2-c9d0-401d-bc87-c429c5efe755 None admin] PATH_INFO: `/v3/users/945ae273f8ef413cad49cd7331140c10` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.729731 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c62b2-c9d0-401d-bc87-c429c5efe755 None admin] RBAC: Authorizing `identity:get_user(user_id=945ae273f8ef413cad49cd7331140c10)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.731227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c62b2-c9d0-401d-bc87-c429c5efe755 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.732891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 42/84] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:42 2022] GET /identity/v3/users/945ae273f8ef413cad49cd7331140c10 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:42.733631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c62b2-c9d0-401d-bc87-c429c5efe755 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.811105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:42.815797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f868b0a-3f0e-488c-b90a-6d3b052805b1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.855349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c62b2-c9d0-401d-bc87-c429c5efe755 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:42.855637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a25572-bcb6-4888-99a6-456fcd795d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.856355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a25572-bcb6-4888-99a6-456fcd795d30 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.856639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a25572-bcb6-4888-99a6-456fcd795d30 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.856954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a25572-bcb6-4888-99a6-456fcd795d30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.890703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd21c6c-dff5-41a9-b338-44f19b197c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:42.892453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd21c6c-dff5-41a9-b338-44f19b197c78 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:42.892882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd21c6c-dff5-41a9-b338-44f19b197c78 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:42.893262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd21c6c-dff5-41a9-b338-44f19b197c78 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:42.894408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd21c6c-dff5-41a9-b338-44f19b197c78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.896513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd21c6c-dff5-41a9-b338-44f19b197c78 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.922801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 43/85] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:39:42 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:42.923830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd21c6c-dff5-41a9-b338-44f19b197c78 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:42.941306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a25572-bcb6-4888-99a6-456fcd795d30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:42.942952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a25572-bcb6-4888-99a6-456fcd795d30 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:42.973948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:43.056738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd21c6c-dff5-41a9-b338-44f19b197c78 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:43.069206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:43.081134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 43/86] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:43.081134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a25572-bcb6-4888-99a6-456fcd795d30 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:43.105192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be395135-b782-46c0-bf27-b55fb6c7ecea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:43.106346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be395135-b782-46c0-bf27-b55fb6c7ecea None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:43.106566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be395135-b782-46c0-bf27-b55fb6c7ecea None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:43.106854 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be395135-b782-46c0-bf27-b55fb6c7ecea None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:43.108250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be395135-b782-46c0-bf27-b55fb6c7ecea None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:43.109742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be395135-b782-46c0-bf27-b55fb6c7ecea None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:43.111621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 44/87] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:42 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:43.112264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be395135-b782-46c0-bf27-b55fb6c7ecea None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:43.205411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a25572-bcb6-4888-99a6-456fcd795d30 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:43.250818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a053a85-184b-4d8b-9c89-e3a7d966f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:43.252195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a053a85-184b-4d8b-9c89-e3a7d966f023 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:43.253823 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a053a85-184b-4d8b-9c89-e3a7d966f023 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:43.253823 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a053a85-184b-4d8b-9c89-e3a7d966f023 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:43.254480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a053a85-184b-4d8b-9c89-e3a7d966f023 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:43.256449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a053a85-184b-4d8b-9c89-e3a7d966f023 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:43.258625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 44/88] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:43 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:43.259094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be395135-b782-46c0-bf27-b55fb6c7ecea None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:43.259258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a053a85-184b-4d8b-9c89-e3a7d966f023 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:43.267693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:43.308422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89366925-5597-4090-9902-cb5a69fe70ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:43.309890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89366925-5597-4090-9902-cb5a69fe70ab None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:43.310431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89366925-5597-4090-9902-cb5a69fe70ab None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:43.310911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89366925-5597-4090-9902-cb5a69fe70ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:43.343488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:43.382616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a053a85-184b-4d8b-9c89-e3a7d966f023 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:43.398212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89366925-5597-4090-9902-cb5a69fe70ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:43.401230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89366925-5597-4090-9902-cb5a69fe70ab None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:43.422013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf167c00-161b-4fe4-ab8b-0840f08b3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:43.423284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167c00-161b-4fe4-ab8b-0840f08b3cd7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:43.423758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167c00-161b-4fe4-ab8b-0840f08b3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:43.424216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167c00-161b-4fe4-ab8b-0840f08b3cd7 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:43.425655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf167c00-161b-4fe4-ab8b-0840f08b3cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:43.427358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf167c00-161b-4fe4-ab8b-0840f08b3cd7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:43.429323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 45/89] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:43 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:43.430325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf167c00-161b-4fe4-ab8b-0840f08b3cd7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:43.546631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 45/90] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:43.546631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89366925-5597-4090-9902-cb5a69fe70ab None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:43.550400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:43.554417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf167c00-161b-4fe4-ab8b-0840f08b3cd7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:43.603153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b00f21-4e0e-4033-9cf2-8c0107e618c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:43.603832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b00f21-4e0e-4033-9cf2-8c0107e618c7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:43.604115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b00f21-4e0e-4033-9cf2-8c0107e618c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:43.604389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b00f21-4e0e-4033-9cf2-8c0107e618c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:43.608467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:43.687002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89366925-5597-4090-9902-cb5a69fe70ab None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:43.708113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b00f21-4e0e-4033-9cf2-8c0107e618c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:43.709647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b00f21-4e0e-4033-9cf2-8c0107e618c7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:43.726858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:43.727868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:43.728405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:43.728887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:43.729973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] RBAC: Adding query filter params (role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62, user.id=4d94401bc90146c29a9e891bdf7a2ed7) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:43.730728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:43.732516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:43.744773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 46/91] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:43 2022] GET /identity/v3/role_assignments?user.id=4d94401bc90146c29a9e891bdf7a2ed7&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:43.746020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:43.833748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:43.861211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 46/92] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:43.865796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b00f21-4e0e-4033-9cf2-8c0107e618c7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:43.895492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e19c0-ee32-4894-879b-cde157617cea None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:43.909085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:43.940979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:43.942119 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:43.942603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:43.943091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:43.944138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] RBAC: Adding query filter params (role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62, user.id=4d94401bc90146c29a9e891bdf7a2ed7) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:43.944650 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:43.946365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:43.960721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 47/93] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:43 2022] GET /identity/v3/role_assignments?user.id=4d94401bc90146c29a9e891bdf7a2ed7&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:43.961924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:43.987138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b00f21-4e0e-4033-9cf2-8c0107e618c7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:44.036842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4a8a0-b41e-4805-8e16-8f960c66dbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:44.037251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4a8a0-b41e-4805-8e16-8f960c66dbfa None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:44.037537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4a8a0-b41e-4805-8e16-8f960c66dbfa None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:44.037840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4a8a0-b41e-4805-8e16-8f960c66dbfa None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:44.042925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4a8a0-b41e-4805-8e16-8f960c66dbfa None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:44.044472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c4a8a0-b41e-4805-8e16-8f960c66dbfa None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:44.051010 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 47/94] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:43 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:44.052432 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4a8a0-b41e-4805-8e16-8f960c66dbfa None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:44.143455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada887a-95bf-4705-ab43-11d5eb55a178 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:44.176803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4a8a0-b41e-4805-8e16-8f960c66dbfa None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:44.178682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:44.186375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:44.202726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe9d7-a93f-4223-8ef0-5cfe859af3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:44.204068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe9d7-a93f-4223-8ef0-5cfe859af3a2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:44.204503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe9d7-a93f-4223-8ef0-5cfe859af3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:44.204913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbe9d7-a93f-4223-8ef0-5cfe859af3a2 None admin] PATH_INFO: `/v3/projects/495160e79d4f4c7e960ab7d794298e67` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:44.206704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbe9d7-a93f-4223-8ef0-5cfe859af3a2 None admin] RBAC: Authorizing `identity:get_project(project_id=495160e79d4f4c7e960ab7d794298e67)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:44.208475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbe9d7-a93f-4223-8ef0-5cfe859af3a2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:44.218277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 48/95] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:43 2022] GET /identity/v3/projects/495160e79d4f4c7e960ab7d794298e67 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:44.219976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe9d7-a93f-4223-8ef0-5cfe859af3a2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:44.242709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236c615-8f2f-4c36-8e81-3d3807ccff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:44.243254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236c615-8f2f-4c36-8e81-3d3807ccff1a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:44.243544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236c615-8f2f-4c36-8e81-3d3807ccff1a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:44.243822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236c615-8f2f-4c36-8e81-3d3807ccff1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:44.244696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236c615-8f2f-4c36-8e81-3d3807ccff1a None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62, user.id=945ae273f8ef413cad49cd7331140c10) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:44.245015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236c615-8f2f-4c36-8e81-3d3807ccff1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:44.247043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236c615-8f2f-4c36-8e81-3d3807ccff1a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:44.259743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 48/96] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:44 2022] GET /identity/v3/role_assignments?user.id=945ae273f8ef413cad49cd7331140c10&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:44.348188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbe9d7-a93f-4223-8ef0-5cfe859af3a2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:44.392614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cece4f-dac0-46db-a480-c807d39846ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:44.393615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cece4f-dac0-46db-a480-c807d39846ab None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:44.394063 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cece4f-dac0-46db-a480-c807d39846ab None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:44.394539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cece4f-dac0-46db-a480-c807d39846ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:44.395537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cece4f-dac0-46db-a480-c807d39846ab None admin] RBAC: Adding query filter params (role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=495160e79d4f4c7e960ab7d794298e67, user.id=4d94401bc90146c29a9e891bdf7a2ed7) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:44.395988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cece4f-dac0-46db-a480-c807d39846ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:44.397660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cece4f-dac0-46db-a480-c807d39846ab None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:44.406385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 49/97] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:44 2022] GET /identity/v3/role_assignments?user.id=4d94401bc90146c29a9e891bdf7a2ed7&scope.project.id=495160e79d4f4c7e960ab7d794298e67&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:45.444125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219223-17cf-4b4c-b006-02da40d157db None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:45.444125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219223-17cf-4b4c-b006-02da40d157db None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:45.444125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219223-17cf-4b4c-b006-02da40d157db None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:45.444125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 49/98] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:45.450692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01ab6c-b467-41c6-b889-3b7c603add35 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:45.451618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01ab6c-b467-41c6-b889-3b7c603add35 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:45.452150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01ab6c-b467-41c6-b889-3b7c603add35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:45.469741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-fc01ab6c-b467-41c6-b889-3b7c603add35 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:45.471314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fc01ab6c-b467-41c6-b889-3b7c603add35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:45.560873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:45.571202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 50/99] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:45.619359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75219223-17cf-4b4c-b006-02da40d157db None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:45.754542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75219223-17cf-4b4c-b006-02da40d157db None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:45.791442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7be0cd-3481-4ecd-b6d1-76b79a285cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:45.792264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7be0cd-3481-4ecd-b6d1-76b79a285cf3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:45.792596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7be0cd-3481-4ecd-b6d1-76b79a285cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:45.792870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7be0cd-3481-4ecd-b6d1-76b79a285cf3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:45.803801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7be0cd-3481-4ecd-b6d1-76b79a285cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-1f7be0cd-3481-4ecd-b6d1-76b79a285cf3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:45.811915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:45.812750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:45.813479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 50/100] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:45.819047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01ab6c-b467-41c6-b889-3b7c603add35 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:45.984827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01ab6c-b467-41c6-b889-3b7c603add35 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:46.030831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:46.033853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:46.033853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:46.033853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:46.033853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:46.033853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:46.038847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:46.047498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 51/101] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:46.053141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7be0cd-3481-4ecd-b6d1-76b79a285cf3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:46.090266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:46.176997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7be0cd-3481-4ecd-b6d1-76b79a285cf3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:46.218841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f9a98-3a28-47b2-9252-d01ea5963204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:46.219750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9a98-3a28-47b2-9252-d01ea5963204 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:46.220038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9a98-3a28-47b2-9252-d01ea5963204 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:46.220320 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f9a98-3a28-47b2-9252-d01ea5963204 None admin] PATH_INFO: `/v3/users/6ff07eed450c4e5dba0e1a8d8c3ec7bd` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:46.236114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f9a98-3a28-47b2-9252-d01ea5963204 None admin] RBAC: Authorizing `identity:get_user(user_id=6ff07eed450c4e5dba0e1a8d8c3ec7bd)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:46.237642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f9a98-3a28-47b2-9252-d01ea5963204 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:46.239570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 51/102] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:46 2022] GET /identity/v3/users/6ff07eed450c4e5dba0e1a8d8c3ec7bd => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:46.245201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:46.329664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:46.387489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9617d-9149-405d-9523-4923288c68a2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:46.403219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a6b43-bf0b-4000-a47c-ab1cde5acf2c None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:46.403518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a6b43-bf0b-4000-a47c-ab1cde5acf2c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:46.403783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a6b43-bf0b-4000-a47c-ab1cde5acf2c None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:46.405193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 52/103] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:46.415697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd3b2a-be56-4ffe-9583-9df7aa0c658d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:46.415981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd3b2a-be56-4ffe-9583-9df7aa0c658d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:46.416263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd3b2a-be56-4ffe-9583-9df7aa0c658d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:46.438667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-90bd3b2a-be56-4ffe-9583-9df7aa0c658d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:46.441375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-90bd3b2a-be56-4ffe-9583-9df7aa0c658d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:46.456837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58177f-a20a-43a9-b372-8cad12e4d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:46.457513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58177f-a20a-43a9-b372-8cad12e4d292 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:46.457865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58177f-a20a-43a9-b372-8cad12e4d292 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:46.458160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58177f-a20a-43a9-b372-8cad12e4d292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:46.542293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 53/104] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:39:46.563561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58177f-a20a-43a9-b372-8cad12e4d292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:46.565125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58177f-a20a-43a9-b372-8cad12e4d292 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:46.566062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd3b2a-be56-4ffe-9583-9df7aa0c658d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:46.581891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:46.634287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:46.697614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd3b2a-be56-4ffe-9583-9df7aa0c658d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:46.699516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 52/105] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:46.706858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58177f-a20a-43a9-b372-8cad12e4d292 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:46.790485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967634c-daf4-41b5-bde0-4b48e4942659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:46.791776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967634c-daf4-41b5-bde0-4b48e4942659 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:46.792127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967634c-daf4-41b5-bde0-4b48e4942659 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:46.792542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967634c-daf4-41b5-bde0-4b48e4942659 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:46.798774 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967634c-daf4-41b5-bde0-4b48e4942659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:46.801398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967634c-daf4-41b5-bde0-4b48e4942659 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:46.803267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 54/106] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:46.811163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967634c-daf4-41b5-bde0-4b48e4942659 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:46.934079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58177f-a20a-43a9-b372-8cad12e4d292 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:47.000089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:47.002068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbb290-430b-49e0-a50d-4a3ef08573bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:47.003799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb290-430b-49e0-a50d-4a3ef08573bb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.004086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb290-430b-49e0-a50d-4a3ef08573bb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.004374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb290-430b-49e0-a50d-4a3ef08573bb None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.010771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbb290-430b-49e0-a50d-4a3ef08573bb None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:47.012508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbb290-430b-49e0-a50d-4a3ef08573bb None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:47.014382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 53/107] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:46 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:47.020484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbb290-430b-49e0-a50d-4a3ef08573bb None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:47.046301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967634c-daf4-41b5-bde0-4b48e4942659 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:47.074069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:47.142829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333710a-08e4-48ad-a7c8-5c7525cdafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:47.143745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333710a-08e4-48ad-a7c8-5c7525cdafd7 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.144048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333710a-08e4-48ad-a7c8-5c7525cdafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.144350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333710a-08e4-48ad-a7c8-5c7525cdafd7 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.145094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333710a-08e4-48ad-a7c8-5c7525cdafd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:47.146622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333710a-08e4-48ad-a7c8-5c7525cdafd7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:47.191405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 55/108] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:39:46 2022] POST /identity/v3/projects => generated 302 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:39:47.192845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8f68-1f33-4ccd-b0a0-45c641ef7eef None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.193144 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8f68-1f33-4ccd-b0a0-45c641ef7eef None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.193427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8f68-1f33-4ccd-b0a0-45c641ef7eef None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.195062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 56/109] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:47.200069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbb290-430b-49e0-a50d-4a3ef08573bb None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:47.201544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194381-e184-4454-a0d1-fda233551e9d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.201918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194381-e184-4454-a0d1-fda233551e9d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.202349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6194381-e184-4454-a0d1-fda233551e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.258574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:47.258888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e6194381-e184-4454-a0d1-fda233551e9d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:47.259249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.259536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.259862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.260068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e6194381-e184-4454-a0d1-fda233551e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:47.260829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] RBAC: Adding query filter params (role.id=24a12434807d480fafb72632fc2d6756, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62, user.id=6ff07eed450c4e5dba0e1a8d8c3ec7bd) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:47.261270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:47.263411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:47.272282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 54/110] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:47 2022] GET /identity/v3/role_assignments?user.id=6ff07eed450c4e5dba0e1a8d8c3ec7bd&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=24a12434807d480fafb72632fc2d6756 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:47.404103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 57/111] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:47.439883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:47.571142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:47.592257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185ad2-a950-4486-ac75-1618a86931f9 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.592778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bea363-9912-43f3-83c2-fee447819bde None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:47.592885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185ad2-a950-4486-ac75-1618a86931f9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.593111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185ad2-a950-4486-ac75-1618a86931f9 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.595444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 58/112] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:39:47.614217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ac4b6-0994-43fc-9863-08d5af74654e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.614552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ac4b6-0994-43fc-9863-08d5af74654e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.614803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ac4b6-0994-43fc-9863-08d5af74654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.639801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0d410-e366-4099-9ec3-a2ab537c5e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:47.640655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0d410-e366-4099-9ec3-a2ab537c5e67 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.641000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0d410-e366-4099-9ec3-a2ab537c5e67 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.641283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0d410-e366-4099-9ec3-a2ab537c5e67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.651082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-628ac4b6-0994-43fc-9863-08d5af74654e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:47.652522 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-628ac4b6-0994-43fc-9863-08d5af74654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:47.656001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0d410-e366-4099-9ec3-a2ab537c5e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-c0d0d410-e366-4099-9ec3-a2ab537c5e67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:47.664411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:47.665538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:47.666358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 55/113] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:47.671484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eae6f-33d7-493c-9cab-6907055e1cb1 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.671827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eae6f-33d7-493c-9cab-6907055e1cb1 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.675896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088eae6f-33d7-493c-9cab-6907055e1cb1 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.676342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 56/114] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:47.680788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eae6f-33d7-493c-9cab-6907055e1cb1 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:47.759765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:47.773363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 59/115] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:39:47.775318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456f89a-aee4-44fd-864c-7df6d24de050 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.775758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456f89a-aee4-44fd-864c-7df6d24de050 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.776207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456f89a-aee4-44fd-864c-7df6d24de050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.809148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5456f89a-aee4-44fd-864c-7df6d24de050 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:47.810676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5456f89a-aee4-44fd-864c-7df6d24de050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:47.836033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088eae6f-33d7-493c-9cab-6907055e1cb1 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:47.881801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:47.882701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:47.883092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:47.883464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:47.889839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] RBAC: Adding query filter params (name=member) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:47.889839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:47.890799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:47.904673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 57/116] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:47.906091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:47.924337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:47.997227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 60/117] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:47.998858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456f89a-aee4-44fd-864c-7df6d24de050 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:48.033809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661282d7-fc54-447b-80f3-bf45002725ec None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:48.075029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b35ff-a4a9-49aa-b3ff-96ce0e568113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:48.075863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b35ff-a4a9-49aa-b3ff-96ce0e568113 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.076149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b35ff-a4a9-49aa-b3ff-96ce0e568113 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.076425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b35ff-a4a9-49aa-b3ff-96ce0e568113 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.090947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b35ff-a4a9-49aa-b3ff-96ce0e568113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-132b35ff-a4a9-49aa-b3ff-96ce0e568113 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:48.099495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:48.100403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:48.101154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 58/118] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:39:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:48.101767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b35ff-a4a9-49aa-b3ff-96ce0e568113 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:48.192932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:48.233246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b35ff-a4a9-49aa-b3ff-96ce0e568113 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:48.275839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b968873-d719-48f5-b23c-14def08c583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:48.276680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b968873-d719-48f5-b23c-14def08c583d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.277092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b968873-d719-48f5-b23c-14def08c583d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.277372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b968873-d719-48f5-b23c-14def08c583d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.299332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b968873-d719-48f5-b23c-14def08c583d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-9b968873-d719-48f5-b23c-14def08c583d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:48.307583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:48.308619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:48.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 59/119] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:39:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:48.322195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b968873-d719-48f5-b23c-14def08c583d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:48.342753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456f89a-aee4-44fd-864c-7df6d24de050 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:48.399137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:48.418550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b56c9d-c748-4498-8592-d849ee9e6440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:48.419474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b56c9d-c748-4498-8592-d849ee9e6440 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.419760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b56c9d-c748-4498-8592-d849ee9e6440 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.420035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b56c9d-c748-4498-8592-d849ee9e6440 None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.424467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b56c9d-c748-4498-8592-d849ee9e6440 None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:48.427904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b56c9d-c748-4498-8592-d849ee9e6440 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:48.429708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 61/120] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:47 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:48.430695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b56c9d-c748-4498-8592-d849ee9e6440 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:48.505673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b968873-d719-48f5-b23c-14def08c583d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:48.545809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:48.554222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:48.554916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.555201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.555482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.556252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:48.556564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:48.558034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:48.570589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 60/121] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:48.571271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:48.674117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b56c9d-c748-4498-8592-d849ee9e6440 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:48.702459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d004ad3-d719-4e66-a71d-20b8dc3d9e9b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:48.710660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8e33a-1b18-495f-98b8-432132ad55d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:48.711363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8e33a-1b18-495f-98b8-432132ad55d1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.711654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8e33a-1b18-495f-98b8-432132ad55d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.711933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8e33a-1b18-495f-98b8-432132ad55d1 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.712710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8e33a-1b18-495f-98b8-432132ad55d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:48.713022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8e33a-1b18-495f-98b8-432132ad55d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:48.714576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8e33a-1b18-495f-98b8-432132ad55d1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:48.724085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 62/122] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:39:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:48.725273 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44767568-e005-4914-b4d3-a6b5250d476a None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.725570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44767568-e005-4914-b4d3-a6b5250d476a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.731290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44767568-e005-4914-b4d3-a6b5250d476a None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.733226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 63/123] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:48.734136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44767568-e005-4914-b4d3-a6b5250d476a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:48.747006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8823935-89bf-4056-92fb-d7e42467c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:48.747908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8823935-89bf-4056-92fb-d7e42467c835 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.748191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8823935-89bf-4056-92fb-d7e42467c835 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.748473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8823935-89bf-4056-92fb-d7e42467c835 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.749735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8823935-89bf-4056-92fb-d7e42467c835 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:48.751387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8823935-89bf-4056-92fb-d7e42467c835 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:48.753338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 61/124] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:48 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:48.754071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8823935-89bf-4056-92fb-d7e42467c835 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:48.804338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:48.827321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:48.855155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44767568-e005-4914-b4d3-a6b5250d476a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:48.878760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8823935-89bf-4056-92fb-d7e42467c835 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:48.893124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549807e6-c556-4309-8ed0-58543a83b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:48.894085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549807e6-c556-4309-8ed0-58543a83b720 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.894564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549807e6-c556-4309-8ed0-58543a83b720 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.894666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549807e6-c556-4309-8ed0-58543a83b720 None admin] PATH_INFO: `/v3/users/945ae273f8ef413cad49cd7331140c10` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.895948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549807e6-c556-4309-8ed0-58543a83b720 None admin] RBAC: Authorizing `identity:get_user(user_id=945ae273f8ef413cad49cd7331140c10)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:48.897491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549807e6-c556-4309-8ed0-58543a83b720 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:48.899784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 64/125] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:48 2022] GET /identity/v3/users/945ae273f8ef413cad49cd7331140c10 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:48.904954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67eca-4574-4ee9-a395-da890267bd2b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.904954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67eca-4574-4ee9-a395-da890267bd2b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.904954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67eca-4574-4ee9-a395-da890267bd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.921603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d9a67eca-4574-4ee9-a395-da890267bd2b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:48.922887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d9a67eca-4574-4ee9-a395-da890267bd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:48.932162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867487e-3014-4257-af5a-d36f8eec4c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:48.933617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867487e-3014-4257-af5a-d36f8eec4c42 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:48.934138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867487e-3014-4257-af5a-d36f8eec4c42 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:48.934748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867487e-3014-4257-af5a-d36f8eec4c42 None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:48.936326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867487e-3014-4257-af5a-d36f8eec4c42 None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:48.938073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867487e-3014-4257-af5a-d36f8eec4c42 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:48.940438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 62/126] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:48 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:48.941442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867487e-3014-4257-af5a-d36f8eec4c42 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:49.008216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 65/127] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:49.014557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a67eca-4574-4ee9-a395-da890267bd2b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:49.077295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867487e-3014-4257-af5a-d36f8eec4c42 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:49.090500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:49.114391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:49.126386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dd3b0-c322-42d3-bf80-207620d60796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:49.128261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dd3b0-c322-42d3-bf80-207620d60796 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:49.128752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dd3b0-c322-42d3-bf80-207620d60796 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:49.129216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483dd3b0-c322-42d3-bf80-207620d60796 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/users/4d94401bc90146c29a9e891bdf7a2ed7/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:49.132153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dd3b0-c322-42d3-bf80-207620d60796 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a4deb4f0ae34b60b855c91ae5f02d62, user_id=4d94401bc90146c29a9e891bdf7a2ed7, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:49.134163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483dd3b0-c322-42d3-bf80-207620d60796 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:49.151901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a67eca-4574-4ee9-a395-da890267bd2b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:49.157430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 63/128] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:39:48 2022] PUT /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/users/4d94401bc90146c29a9e891bdf7a2ed7/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:39:49.158686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dd3b0-c322-42d3-bf80-207620d60796 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:49.195729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6c769-e8fd-43fa-bea9-af94eef52246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:49.196665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6c769-e8fd-43fa-bea9-af94eef52246 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:49.197029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6c769-e8fd-43fa-bea9-af94eef52246 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:49.197430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6c769-e8fd-43fa-bea9-af94eef52246 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:49.199378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6c769-e8fd-43fa-bea9-af94eef52246 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:49.201043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6c769-e8fd-43fa-bea9-af94eef52246 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:49.203111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 66/129] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:49 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:49.204090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6c769-e8fd-43fa-bea9-af94eef52246 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:49.383952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6c769-e8fd-43fa-bea9-af94eef52246 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:49.398327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483dd3b0-c322-42d3-bf80-207620d60796 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:49.420127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:49.453179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354fb49-cadb-4074-b5a0-a3f7b5588734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:49.454431 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354fb49-cadb-4074-b5a0-a3f7b5588734 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:49.454944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354fb49-cadb-4074-b5a0-a3f7b5588734 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:49.455416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354fb49-cadb-4074-b5a0-a3f7b5588734 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:49.464952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b5b99-dd88-4211-8aff-440235ad96c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:49.466080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b5b99-dd88-4211-8aff-440235ad96c2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:49.466646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b5b99-dd88-4211-8aff-440235ad96c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:49.467025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b5b99-dd88-4211-8aff-440235ad96c2 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:49.469856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b5b99-dd88-4211-8aff-440235ad96c2 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:49.472363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b5b99-dd88-4211-8aff-440235ad96c2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:49.478931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354fb49-cadb-4074-b5a0-a3f7b5588734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:49.481169 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 64/130] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:49 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:49.482151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b5b99-dd88-4211-8aff-440235ad96c2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-6354fb49-cadb-4074-b5a0-a3f7b5588734 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:49.492116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:49.493023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:49.493766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:49.496693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 67/131] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:49.498137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354fb49-cadb-4074-b5a0-a3f7b5588734 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:49.593167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:49.643707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b5b99-dd88-4211-8aff-440235ad96c2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:49.683707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b62d01-9b04-4e6d-a189-bfa121472a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:49.684921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b62d01-9b04-4e6d-a189-bfa121472a1f None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:49.685347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b62d01-9b04-4e6d-a189-bfa121472a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:49.685760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b62d01-9b04-4e6d-a189-bfa121472a1f None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/users/945ae273f8ef413cad49cd7331140c10/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:49.688143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b62d01-9b04-4e6d-a189-bfa121472a1f None admin] RBAC: Authorizing `identity:create_grant(project_id=0a4deb4f0ae34b60b855c91ae5f02d62, user_id=945ae273f8ef413cad49cd7331140c10, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:49.689952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b62d01-9b04-4e6d-a189-bfa121472a1f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:49.709627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 65/132] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:39:49 2022] PUT /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/users/945ae273f8ef413cad49cd7331140c10/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:39:49.710800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b62d01-9b04-4e6d-a189-bfa121472a1f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:49.737559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354fb49-cadb-4074-b5a0-a3f7b5588734 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:49.815181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb276da-8c6c-46e4-9268-2478854894ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:49.816129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb276da-8c6c-46e4-9268-2478854894ae None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:49.816417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb276da-8c6c-46e4-9268-2478854894ae None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:49.816700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb276da-8c6c-46e4-9268-2478854894ae None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/users/4d94401bc90146c29a9e891bdf7a2ed7/roles/a505b2a721044e6ea8312ef6ffdc1c9f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:49.819088 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb276da-8c6c-46e4-9268-2478854894ae None admin] RBAC: Authorizing `identity:create_grant(project_id=0a4deb4f0ae34b60b855c91ae5f02d62, user_id=4d94401bc90146c29a9e891bdf7a2ed7, role_id=a505b2a721044e6ea8312ef6ffdc1c9f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:49.820593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb276da-8c6c-46e4-9268-2478854894ae None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:49.853851 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 68/133] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:39:49 2022] PUT /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/users/4d94401bc90146c29a9e891bdf7a2ed7/roles/a505b2a721044e6ea8312ef6ffdc1c9f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:39:49.861855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b62d01-9b04-4e6d-a189-bfa121472a1f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:49.883847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:49.909442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a3f61-4f5b-4a14-8e9c-89bd4c107424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:49.910444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3f61-4f5b-4a14-8e9c-89bd4c107424 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:49.910889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3f61-4f5b-4a14-8e9c-89bd4c107424 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:49.911314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3f61-4f5b-4a14-8e9c-89bd4c107424 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:49.912287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3f61-4f5b-4a14-8e9c-89bd4c107424 None admin] RBAC: Adding query filter params (name=member) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:49.912738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3f61-4f5b-4a14-8e9c-89bd4c107424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:49.914446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3f61-4f5b-4a14-8e9c-89bd4c107424 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:49.924813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 66/134] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:49.930867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb276da-8c6c-46e4-9268-2478854894ae None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:49.996656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:50.061590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20ecdd-7c3a-405e-ae11-525bdb459b7b None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.062310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20ecdd-7c3a-405e-ae11-525bdb459b7b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.062813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20ecdd-7c3a-405e-ae11-525bdb459b7b None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.064721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 67/135] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:39:50.070741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529bccd-f063-4179-9ec7-186ede65d8bc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.071435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529bccd-f063-4179-9ec7-186ede65d8bc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.071909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529bccd-f063-4179-9ec7-186ede65d8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.081308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb276da-8c6c-46e4-9268-2478854894ae None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:50.110963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2529bccd-f063-4179-9ec7-186ede65d8bc None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:50.112615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2529bccd-f063-4179-9ec7-186ede65d8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:50.119544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3513e58-1295-4575-9ca5-b7aa24b6bcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:50.120770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3513e58-1295-4575-9ca5-b7aa24b6bcc3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.121060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3513e58-1295-4575-9ca5-b7aa24b6bcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.121335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3513e58-1295-4575-9ca5-b7aa24b6bcc3 None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.123217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3513e58-1295-4575-9ca5-b7aa24b6bcc3 None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:50.124786 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3513e58-1295-4575-9ca5-b7aa24b6bcc3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:50.126855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 69/136] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:49 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:50.133102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3513e58-1295-4575-9ca5-b7aa24b6bcc3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:50.212583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 68/137] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:39:50.251508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3513e58-1295-4575-9ca5-b7aa24b6bcc3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:50.259334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529bccd-f063-4179-9ec7-186ede65d8bc None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:50.261551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:50.289696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3758c-7327-44c6-abaa-fba5188c87c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:50.290640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3758c-7327-44c6-abaa-fba5188c87c9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.290927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3758c-7327-44c6-abaa-fba5188c87c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.291207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3758c-7327-44c6-abaa-fba5188c87c9 None admin] PATH_INFO: `/v3/projects/495160e79d4f4c7e960ab7d794298e67` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.292822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3758c-7327-44c6-abaa-fba5188c87c9 None admin] RBAC: Authorizing `identity:get_project(project_id=495160e79d4f4c7e960ab7d794298e67)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:50.294477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3758c-7327-44c6-abaa-fba5188c87c9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:50.301798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 70/138] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:50 2022] GET /identity/v3/projects/495160e79d4f4c7e960ab7d794298e67 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:50.302326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3758c-7327-44c6-abaa-fba5188c87c9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:50.422473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3758c-7327-44c6-abaa-fba5188c87c9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:50.455159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529bccd-f063-4179-9ec7-186ede65d8bc None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:50.466152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:50.467002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63fadd-756a-4c44-9720-baa36d880bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:50.468188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63fadd-756a-4c44-9720-baa36d880bc1 None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.468662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63fadd-756a-4c44-9720-baa36d880bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.469059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63fadd-756a-4c44-9720-baa36d880bc1 None admin] PATH_INFO: `/v3/projects/495160e79d4f4c7e960ab7d794298e67/users/4d94401bc90146c29a9e891bdf7a2ed7/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.471902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63fadd-756a-4c44-9720-baa36d880bc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=495160e79d4f4c7e960ab7d794298e67, user_id=4d94401bc90146c29a9e891bdf7a2ed7, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:50.473603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63fadd-756a-4c44-9720-baa36d880bc1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:50.489503 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 71/139] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:39:50 2022] PUT /identity/v3/projects/495160e79d4f4c7e960ab7d794298e67/users/4d94401bc90146c29a9e891bdf7a2ed7/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:39:50.491529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e239-6790-4d60-a55c-761672ba8280 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.491828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e239-6790-4d60-a55c-761672ba8280 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.492226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e239-6790-4d60-a55c-761672ba8280 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.493668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 72/140] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:50.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2fc5-209d-4f59-9a1b-a8a1a5446569 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.500192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2fc5-209d-4f59-9a1b-a8a1a5446569 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.500547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf2fc5-209d-4f59-9a1b-a8a1a5446569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.522307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5acf2fc5-209d-4f59-9a1b-a8a1a5446569 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:50.524549 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5acf2fc5-209d-4f59-9a1b-a8a1a5446569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:50.566170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:50.591588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7a63c-eca7-4eff-9256-c1922506be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:50.592417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7a63c-eca7-4eff-9256-c1922506be15 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.592699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7a63c-eca7-4eff-9256-c1922506be15 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.592974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7a63c-eca7-4eff-9256-c1922506be15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.594218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7a63c-eca7-4eff-9256-c1922506be15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:50.595747 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7a63c-eca7-4eff-9256-c1922506be15 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:50.597667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 69/141] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:50.606873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7a63c-eca7-4eff-9256-c1922506be15 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:50.612274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 73/142] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:39:50.639515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf2fc5-209d-4f59-9a1b-a8a1a5446569 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:50.762125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf2fc5-209d-4f59-9a1b-a8a1a5446569 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:50.804814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8c7cb-ee36-4725-965f-22e82ff9684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:50.805620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8c7cb-ee36-4725-965f-22e82ff9684f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.805947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8c7cb-ee36-4725-965f-22e82ff9684f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.806235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8c7cb-ee36-4725-965f-22e82ff9684f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:50.819947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8c7cb-ee36-4725-965f-22e82ff9684f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-09c8c7cb-ee36-4725-965f-22e82ff9684f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:50.828843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:50.829748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:50.830531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 74/143] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:39:50.836971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8c7cb-ee36-4725-965f-22e82ff9684f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:50.886828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:50.955418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8c7cb-ee36-4725-965f-22e82ff9684f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:50.955747 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7a63c-eca7-4eff-9256-c1922506be15 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:50.997763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:50.998852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:50.999152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:50.999452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:51.000322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:51.000711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:51.002342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:51.013134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 75/144] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:51.018678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:51.069867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234871f-233d-4e2c-9fbf-2d07b773ac1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:51.070480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234871f-233d-4e2c-9fbf-2d07b773ac1f None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:51.070773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234871f-233d-4e2c-9fbf-2d07b773ac1f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:51.071049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234871f-233d-4e2c-9fbf-2d07b773ac1f None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:51.071772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234871f-233d-4e2c-9fbf-2d07b773ac1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:51.073235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234871f-233d-4e2c-9fbf-2d07b773ac1f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:51.129294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:51.137099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 70/145] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:39:50 2022] POST /identity/v3/users => generated 283 bytes in 531 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:51.145451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de03847-f4eb-43e2-b157-8daf0325a4a1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:51.191105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf81c49-a2b8-4e0c-91a6-5e1954e3e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:51.192018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81c49-a2b8-4e0c-91a6-5e1954e3e656 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:51.192302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81c49-a2b8-4e0c-91a6-5e1954e3e656 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:51.192579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf81c49-a2b8-4e0c-91a6-5e1954e3e656 None admin] PATH_INFO: `/v3/users/6ff07eed450c4e5dba0e1a8d8c3ec7bd` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:51.193845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf81c49-a2b8-4e0c-91a6-5e1954e3e656 None admin] RBAC: Authorizing `identity:get_user(user_id=6ff07eed450c4e5dba0e1a8d8c3ec7bd)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:51.195608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf81c49-a2b8-4e0c-91a6-5e1954e3e656 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:51.197648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 76/146] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:51 2022] GET /identity/v3/users/6ff07eed450c4e5dba0e1a8d8c3ec7bd => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:51.203866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234871f-233d-4e2c-9fbf-2d07b773ac1f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:51.254375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:51.356404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234871f-233d-4e2c-9fbf-2d07b773ac1f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:51.397095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0ff0c-e73e-4ff3-9024-d1e8e5b185a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:51.398027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0ff0c-e73e-4ff3-9024-d1e8e5b185a5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:51.398313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0ff0c-e73e-4ff3-9024-d1e8e5b185a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:51.398590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0ff0c-e73e-4ff3-9024-d1e8e5b185a5 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:51.400302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0ff0c-e73e-4ff3-9024-d1e8e5b185a5 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:51.402043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0ff0c-e73e-4ff3-9024-d1e8e5b185a5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:51.404159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 71/147] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:51 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:51.410183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf81c49-a2b8-4e0c-91a6-5e1954e3e656 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:51.551350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf81c49-a2b8-4e0c-91a6-5e1954e3e656 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:51.589591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7d5df-a1af-4379-855c-79ac74715800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:51.590898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7d5df-a1af-4379-855c-79ac74715800 None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:51.591184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7d5df-a1af-4379-855c-79ac74715800 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:51.591463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7d5df-a1af-4379-855c-79ac74715800 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/users/6ff07eed450c4e5dba0e1a8d8c3ec7bd/roles/24a12434807d480fafb72632fc2d6756` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:51.593729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7d5df-a1af-4379-855c-79ac74715800 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a4deb4f0ae34b60b855c91ae5f02d62, user_id=6ff07eed450c4e5dba0e1a8d8c3ec7bd, role_id=24a12434807d480fafb72632fc2d6756)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:51.595309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7d5df-a1af-4379-855c-79ac74715800 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:51.611240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 77/148] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:39:51 2022] PUT /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/users/6ff07eed450c4e5dba0e1a8d8c3ec7bd/roles/24a12434807d480fafb72632fc2d6756 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:39:52.396215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67f2db-fdc9-4e33-8eb3-2aa587f5053c None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:52.396710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67f2db-fdc9-4e33-8eb3-2aa587f5053c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:52.396782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67f2db-fdc9-4e33-8eb3-2aa587f5053c None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:52.398463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 72/149] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:52.404396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab71133-a3f4-4799-a0a5-66eb323c9577 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:52.405270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab71133-a3f4-4799-a0a5-66eb323c9577 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:52.405741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab71133-a3f4-4799-a0a5-66eb323c9577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:52.430839 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-bab71133-a3f4-4799-a0a5-66eb323c9577 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:52.432376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bab71133-a3f4-4799-a0a5-66eb323c9577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:52.444801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:52.516086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 78/150] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:52.531567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e91d4-9909-46ac-85d3-903345d21bc4 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:52.532020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e91d4-9909-46ac-85d3-903345d21bc4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:52.532378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e91d4-9909-46ac-85d3-903345d21bc4 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:52.534243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 73/151] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:52.540278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cb69a-1465-45b8-94c4-5b5d7860237a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:52.540575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cb69a-1465-45b8-94c4-5b5d7860237a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:52.540857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cb69a-1465-45b8-94c4-5b5d7860237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:52.550554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e91d4-9909-46ac-85d3-903345d21bc4 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:52.573138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e02cb69a-1465-45b8-94c4-5b5d7860237a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:52.574698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e02cb69a-1465-45b8-94c4-5b5d7860237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:52.709015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 79/152] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:52.725302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:52.730122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e91d4-9909-46ac-85d3-903345d21bc4 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:52.758323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cb69a-1465-45b8-94c4-5b5d7860237a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:52.793803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792cfd1-c950-4eff-a014-ddc1cc1dd063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:52.794555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792cfd1-c950-4eff-a014-ddc1cc1dd063 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:52.794838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792cfd1-c950-4eff-a014-ddc1cc1dd063 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:52.795115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792cfd1-c950-4eff-a014-ddc1cc1dd063 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:52.822453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792cfd1-c950-4eff-a014-ddc1cc1dd063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-8792cfd1-c950-4eff-a014-ddc1cc1dd063 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:52.832427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:52.833369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:52.834134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 74/153] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:39:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:39:52.864456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792cfd1-c950-4eff-a014-ddc1cc1dd063 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:52.891899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:52.930596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cb69a-1465-45b8-94c4-5b5d7860237a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:52.988759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84188df8-4c8b-4cf3-86a4-3e73ae0978e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:52.989596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84188df8-4c8b-4cf3-86a4-3e73ae0978e3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:52.989906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84188df8-4c8b-4cf3-86a4-3e73ae0978e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:52.990183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84188df8-4c8b-4cf3-86a4-3e73ae0978e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.004544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792cfd1-c950-4eff-a014-ddc1cc1dd063 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:53.017071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84188df8-4c8b-4cf3-86a4-3e73ae0978e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-84188df8-4c8b-4cf3-86a4-3e73ae0978e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:53.025296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:53.027477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:53.029389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:53.035113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 80/154] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:53.042999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84188df8-4c8b-4cf3-86a4-3e73ae0978e3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:53.053906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:53.054476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:53.054758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:53.055035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.055808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:53.056114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:53.057522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:53.071515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 75/155] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:53.077467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:53.118148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:53.199153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84188df8-4c8b-4cf3-86a4-3e73ae0978e3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:53.255454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:53.256160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:53.256449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:53.256728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.257498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:53.257844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:53.259281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:53.262029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:53.274684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed8b2a-424a-472e-bae4-30a4f6f277a7 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:53.277584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 81/156] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:53.283067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:53.335740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a575-f85f-4d48-aac2-6f08e78a6955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:53.336560 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a575-f85f-4d48-aac2-6f08e78a6955 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:53.336853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a575-f85f-4d48-aac2-6f08e78a6955 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:53.337172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a575-f85f-4d48-aac2-6f08e78a6955 None admin] PATH_INFO: `/v3/users/945ae273f8ef413cad49cd7331140c10` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.338710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a575-f85f-4d48-aac2-6f08e78a6955 None admin] RBAC: Authorizing `identity:get_user(user_id=945ae273f8ef413cad49cd7331140c10)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:53.340368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a575-f85f-4d48-aac2-6f08e78a6955 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:53.342419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 76/157] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:53 2022] GET /identity/v3/users/945ae273f8ef413cad49cd7331140c10 => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:53.347958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a575-f85f-4d48-aac2-6f08e78a6955 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:53.394643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6e11f-5f60-42d1-8b0d-c9bdaf2cb751 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:53.441224 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51cec9-6ae4-4252-84df-6c8cd71f9b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:53.442168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51cec9-6ae4-4252-84df-6c8cd71f9b04 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:53.442448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51cec9-6ae4-4252-84df-6c8cd71f9b04 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:53.442725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51cec9-6ae4-4252-84df-6c8cd71f9b04 None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.446508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51cec9-6ae4-4252-84df-6c8cd71f9b04 None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:53.448018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51cec9-6ae4-4252-84df-6c8cd71f9b04 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:53.450319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 82/158] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:53 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:53.456545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51cec9-6ae4-4252-84df-6c8cd71f9b04 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:53.470222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:53.483370 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a575-f85f-4d48-aac2-6f08e78a6955 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:53.522881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c9f20-ffd1-40e4-a8cd-0c8f2430a5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:53.524211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c9f20-ffd1-40e4-a8cd-0c8f2430a5a7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:53.524665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c9f20-ffd1-40e4-a8cd-0c8f2430a5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:53.525139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c9f20-ffd1-40e4-a8cd-0c8f2430a5a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.617038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:53.619680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c9f20-ffd1-40e4-a8cd-0c8f2430a5a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:53.621979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c9f20-ffd1-40e4-a8cd-0c8f2430a5a7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:53.721643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51cec9-6ae4-4252-84df-6c8cd71f9b04 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:53.732196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:53.745545 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 77/159] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 15:39:53.751820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c9f20-ffd1-40e4-a8cd-0c8f2430a5a7 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:53.802865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905720a-5d66-4cba-b784-052d19901e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:53.803545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905720a-5d66-4cba-b784-052d19901e65 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:53.803827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905720a-5d66-4cba-b784-052d19901e65 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:53.804103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905720a-5d66-4cba-b784-052d19901e65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.868233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c9f20-ffd1-40e4-a8cd-0c8f2430a5a7 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:53.915009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a1de7-5064-48c7-b5c1-7d9308b89544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:53.916254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a1de7-5064-48c7-b5c1-7d9308b89544 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:53.916709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a1de7-5064-48c7-b5c1-7d9308b89544 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:53.917359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a1de7-5064-48c7-b5c1-7d9308b89544 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.919050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a1de7-5064-48c7-b5c1-7d9308b89544 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:53.921108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a1de7-5064-48c7-b5c1-7d9308b89544 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:53.923096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 78/160] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:53 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:53.924505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd9a9-6469-4990-8a4c-e190e78ad9fe None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:53.924968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd9a9-6469-4990-8a4c-e190e78ad9fe None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:53.925471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbd9a9-6469-4990-8a4c-e190e78ad9fe None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:53.931834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 79/161] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:53.934612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd9a9-6469-4990-8a4c-e190e78ad9fe None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:53.959718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905720a-5d66-4cba-b784-052d19901e65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:53.961326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905720a-5d66-4cba-b784-052d19901e65 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:54.038256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:54.074203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbd9a9-6469-4990-8a4c-e190e78ad9fe None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:54.115986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0afb-5cf8-4563-a8ee-5571abf5f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:54.116992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0afb-5cf8-4563-a8ee-5571abf5f4fa None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.117444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0afb-5cf8-4563-a8ee-5571abf5f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.117916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0afb-5cf8-4563-a8ee-5571abf5f4fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.119008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac0afb-5cf8-4563-a8ee-5571abf5f4fa None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62, user.id=945ae273f8ef413cad49cd7331140c10) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:54.119735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac0afb-5cf8-4563-a8ee-5571abf5f4fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:54.121403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac0afb-5cf8-4563-a8ee-5571abf5f4fa None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:54.128603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 80/162] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:53 2022] GET /identity/v3/role_assignments?user.id=945ae273f8ef413cad49cd7331140c10&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:54.135014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298867a-e544-40c7-8157-a6afd5ade6bc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.135585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298867a-e544-40c7-8157-a6afd5ade6bc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.136100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298867a-e544-40c7-8157-a6afd5ade6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.161028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0298867a-e544-40c7-8157-a6afd5ade6bc None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:54.162739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0298867a-e544-40c7-8157-a6afd5ade6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:54.196802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:54.265320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 81/163] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:54.267068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d97e08-44ec-49e7-8319-47bf99f996d9 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.267582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d97e08-44ec-49e7-8319-47bf99f996d9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.268056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d97e08-44ec-49e7-8319-47bf99f996d9 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.270407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 82/164] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:54.276814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81679b-7132-4fcf-9921-8a73dcb2f682 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.277337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81679b-7132-4fcf-9921-8a73dcb2f682 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.277802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81679b-7132-4fcf-9921-8a73dcb2f682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.300295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-5a81679b-7132-4fcf-9921-8a73dcb2f682 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:54.302313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5a81679b-7132-4fcf-9921-8a73dcb2f682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:54.328539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 83/165] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 872 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:54.329426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905720a-5d66-4cba-b784-052d19901e65 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:54.374284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:54.385870 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 83/166] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:39:54.387462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81679b-7132-4fcf-9921-8a73dcb2f682 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:54.511174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81679b-7132-4fcf-9921-8a73dcb2f682 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:54.536854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905720a-5d66-4cba-b784-052d19901e65 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:54.565976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f439db-e6a2-4226-b15d-67fe0ed006b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:54.567441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f439db-e6a2-4226-b15d-67fe0ed006b4 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.568101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f439db-e6a2-4226-b15d-67fe0ed006b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.568548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f439db-e6a2-4226-b15d-67fe0ed006b4 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.570499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f439db-e6a2-4226-b15d-67fe0ed006b4 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:54.572882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f439db-e6a2-4226-b15d-67fe0ed006b4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:54.575645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 84/167] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:54 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:54.580861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f439db-e6a2-4226-b15d-67fe0ed006b4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:54.583657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385877d4-2124-4dae-9799-dd7471bd2e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:54.584773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385877d4-2124-4dae-9799-dd7471bd2e70 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.585247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385877d4-2124-4dae-9799-dd7471bd2e70 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.585843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385877d4-2124-4dae-9799-dd7471bd2e70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.598136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385877d4-2124-4dae-9799-dd7471bd2e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-385877d4-2124-4dae-9799-dd7471bd2e70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:54.607278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:54.608177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:39:54.609215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:54.612193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 84/168] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:39:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:54.613223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385877d4-2124-4dae-9799-dd7471bd2e70 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:54.642185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:54.654002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:54.708624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f439db-e6a2-4226-b15d-67fe0ed006b4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:54.743846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385877d4-2124-4dae-9799-dd7471bd2e70 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:54.749887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bc44b-6c78-47c0-bcdc-4b7099713acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:54.750713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc44b-6c78-47c0-bcdc-4b7099713acb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.751001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc44b-6c78-47c0-bcdc-4b7099713acb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.751278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bc44b-6c78-47c0-bcdc-4b7099713acb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.765240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bc44b-6c78-47c0-bcdc-4b7099713acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-9f8bc44b-6c78-47c0-bcdc-4b7099713acb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:54.775314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:54.776210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:54.777092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 85/169] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:54.778265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bc44b-6c78-47c0-bcdc-4b7099713acb None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:54.789675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:54.790747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.791253 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.791768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.792976 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] RBAC: Adding query filter params (role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62, user.id=4d94401bc90146c29a9e891bdf7a2ed7) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:54.793577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:54.795327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:54.802509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 85/170] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:54 2022] GET /identity/v3/role_assignments?user.id=4d94401bc90146c29a9e891bdf7a2ed7&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:54.803615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:54.898956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bc44b-6c78-47c0-bcdc-4b7099713acb None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:54.912777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:54.965674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:54.967200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:54.967686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:54.968915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:54.969954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:54.970470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:54.972288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:54.984049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 86/171] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:39:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:54.989864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:55.007719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:55.030061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de06b03-50df-472c-a0c4-fc6a86fc3ca1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:55.107981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:55.108683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:55.109021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:55.109304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:55.110107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:55.110427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:55.113118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:55.127122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 86/172] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:55.132386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:55.193725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbbae9-0d0a-4191-b309-2a64222f4a20 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:55.281605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d136b5-652b-4a1b-87f0-2c44223c25cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:55.287672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d136b5-652b-4a1b-87f0-2c44223c25cf None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:55.288336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d136b5-652b-4a1b-87f0-2c44223c25cf None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:55.291088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d136b5-652b-4a1b-87f0-2c44223c25cf None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:55.292658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d136b5-652b-4a1b-87f0-2c44223c25cf None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:55.294677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d136b5-652b-4a1b-87f0-2c44223c25cf None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:55.297696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 87/173] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:54 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:39:55.304993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d136b5-652b-4a1b-87f0-2c44223c25cf None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:55.377570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae405a8-92cd-4f38-b52f-c290ef51f289 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:55.394233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:55.451134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b466185-f5c2-4941-bd8e-e274790aa7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:55.452435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b466185-f5c2-4941-bd8e-e274790aa7f5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:55.452912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b466185-f5c2-4941-bd8e-e274790aa7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:55.453377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b466185-f5c2-4941-bd8e-e274790aa7f5 None admin] PATH_INFO: `/v3/users/4d94401bc90146c29a9e891bdf7a2ed7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:55.459602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b466185-f5c2-4941-bd8e-e274790aa7f5 None admin] RBAC: Authorizing `identity:get_user(user_id=4d94401bc90146c29a9e891bdf7a2ed7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:55.461326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b466185-f5c2-4941-bd8e-e274790aa7f5 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:55.463289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 87/174] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:55 2022] GET /identity/v3/users/4d94401bc90146c29a9e891bdf7a2ed7 => generated 275 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:55.469190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b466185-f5c2-4941-bd8e-e274790aa7f5 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:55.511469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:55.514965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d136b5-652b-4a1b-87f0-2c44223c25cf None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:55.598102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f1a74-94b1-49fc-bcea-51f4f568e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:55.598789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f1a74-94b1-49fc-bcea-51f4f568e50d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:55.599076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f1a74-94b1-49fc-bcea-51f4f568e50d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:55.599355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f1a74-94b1-49fc-bcea-51f4f568e50d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:55.691766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b466185-f5c2-4941-bd8e-e274790aa7f5 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:55.742069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f1a74-94b1-49fc-bcea-51f4f568e50d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:55.743496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f1a74-94b1-49fc-bcea-51f4f568e50d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:55.761296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e8faa-4cfd-4e34-bf74-e51ae8e20e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:55.761999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e8faa-4cfd-4e34-bf74-e51ae8e20e7a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:55.762279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e8faa-4cfd-4e34-bf74-e51ae8e20e7a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:55.762649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e8faa-4cfd-4e34-bf74-e51ae8e20e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:55.835459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:55.863766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e8faa-4cfd-4e34-bf74-e51ae8e20e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:55.865596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e8faa-4cfd-4e34-bf74-e51ae8e20e7a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:55.890199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:55.924148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 88/175] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:55.926739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cec00-0e21-4380-a42d-a74e412fd981 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:55.927232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cec00-0e21-4380-a42d-a74e412fd981 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:55.927982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cec00-0e21-4380-a42d-a74e412fd981 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:55.929902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 89/176] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:55.931496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d1dff-43b8-4f58-9161-61a4f694bf86 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:55.931976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d1dff-43b8-4f58-9161-61a4f694bf86 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:55.932459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d1dff-43b8-4f58-9161-61a4f694bf86 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:55.934448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 90/177] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:55.939245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d1dff-43b8-4f58-9161-61a4f694bf86 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.005871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 88/178] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:56.008029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d386c5-c77a-4666-a239-f54236989621 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.008481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d386c5-c77a-4666-a239-f54236989621 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.009316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d386c5-c77a-4666-a239-f54236989621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.028815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-26d386c5-c77a-4666-a239-f54236989621 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:56.030302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-26d386c5-c77a-4666-a239-f54236989621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:56.071337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d1dff-43b8-4f58-9161-61a4f694bf86 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.107658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 89/179] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:56.109270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644daf-5014-4e98-8e73-966785ee4530 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.109748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644daf-5014-4e98-8e73-966785ee4530 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.110237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3644daf-5014-4e98-8e73-966785ee4530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.111827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c1ee8-2a26-4650-9dfb-d3720ba66dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:56.112806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1ee8-2a26-4650-9dfb-d3720ba66dc8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.113050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1ee8-2a26-4650-9dfb-d3720ba66dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.113334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1ee8-2a26-4650-9dfb-d3720ba66dc8 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.114639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c1ee8-2a26-4650-9dfb-d3720ba66dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:56.116327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c1ee8-2a26-4650-9dfb-d3720ba66dc8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:56.118879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 91/180] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:55 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:56.125177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c1ee8-2a26-4650-9dfb-d3720ba66dc8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.130902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d3644daf-5014-4e98-8e73-966785ee4530 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:56.132079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d3644daf-5014-4e98-8e73-966785ee4530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:56.138297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:56.181920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:56.206330 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 90/181] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:56.207989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441676-85fb-4792-b928-b7425c0dfa3d None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.208469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441676-85fb-4792-b928-b7425c0dfa3d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.208929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441676-85fb-4792-b928-b7425c0dfa3d None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.210430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 91/182] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:56.211259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe441676-85fb-4792-b928-b7425c0dfa3d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.244795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c1ee8-2a26-4650-9dfb-d3720ba66dc8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.282821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff69ff4-d95f-4a29-9d75-40fda617e815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:56.283618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff69ff4-d95f-4a29-9d75-40fda617e815 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.283899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff69ff4-d95f-4a29-9d75-40fda617e815 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.284175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff69ff4-d95f-4a29-9d75-40fda617e815 None admin] PATH_INFO: `/v3/projects/495160e79d4f4c7e960ab7d794298e67` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.285377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff69ff4-d95f-4a29-9d75-40fda617e815 None admin] RBAC: Authorizing `identity:get_project(project_id=495160e79d4f4c7e960ab7d794298e67)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:56.286880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff69ff4-d95f-4a29-9d75-40fda617e815 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:56.293752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 92/183] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/projects/495160e79d4f4c7e960ab7d794298e67 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:56.293752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff69ff4-d95f-4a29-9d75-40fda617e815 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.340337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe441676-85fb-4792-b928-b7425c0dfa3d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.376975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:56.378332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1108dc3-f34f-4523-8767-cff22649546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:56.379344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1108dc3-f34f-4523-8767-cff22649546c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.379720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1108dc3-f34f-4523-8767-cff22649546c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.380163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1108dc3-f34f-4523-8767-cff22649546c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.381015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1108dc3-f34f-4523-8767-cff22649546c None admin] RBAC: Adding query filter params (role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62, user.id=4d94401bc90146c29a9e891bdf7a2ed7) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:56.381325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1108dc3-f34f-4523-8767-cff22649546c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:56.382829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1108dc3-f34f-4523-8767-cff22649546c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:56.389479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 92/184] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/role_assignments?user.id=4d94401bc90146c29a9e891bdf7a2ed7&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:56.395815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cd3ee-49b4-4332-8e2b-fc1cd8afb453 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.396322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cd3ee-49b4-4332-8e2b-fc1cd8afb453 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.397194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cd3ee-49b4-4332-8e2b-fc1cd8afb453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.404680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff69ff4-d95f-4a29-9d75-40fda617e815 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.421362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ca1cd3ee-49b4-4332-8e2b-fc1cd8afb453 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:56.423647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ca1cd3ee-49b4-4332-8e2b-fc1cd8afb453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:56.459951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aaba8-fefd-4a84-b516-76d8e781926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:56.461316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aaba8-fefd-4a84-b516-76d8e781926a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.462316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aaba8-fefd-4a84-b516-76d8e781926a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.462778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7aaba8-fefd-4a84-b516-76d8e781926a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.479745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7aaba8-fefd-4a84-b516-76d8e781926a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:56.481258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-ed7aaba8-fefd-4a84-b516-76d8e781926a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:56.489695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:56.490690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:56.491478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 93/185] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:56.493219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aaba8-fefd-4a84-b516-76d8e781926a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.520180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 93/186] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:56.520832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cd3ee-49b4-4332-8e2b-fc1cd8afb453 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.622851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7aaba8-fefd-4a84-b516-76d8e781926a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.637257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cd3ee-49b4-4332-8e2b-fc1cd8afb453 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.657014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5548c2c-189d-4ec1-98d7-7b9d21de8beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:56.657840 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5548c2c-189d-4ec1-98d7-7b9d21de8beb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.658124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5548c2c-189d-4ec1-98d7-7b9d21de8beb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.658396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5548c2c-189d-4ec1-98d7-7b9d21de8beb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.661027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:56.669713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5548c2c-189d-4ec1-98d7-7b9d21de8beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:56.672267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:56.672938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.673220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.673494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.674376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] RBAC: Adding query filter params (role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=495160e79d4f4c7e960ab7d794298e67, user.id=4d94401bc90146c29a9e891bdf7a2ed7) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:56.674688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:56.676154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f5548c2c-189d-4ec1-98d7-7b9d21de8beb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:56.677438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:56.678458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:39:56.679372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 94/187] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:56.680112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65456cab-df04-455b-ae6a-a919574e39cb None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.680399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65456cab-df04-455b-ae6a-a919574e39cb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.680680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65456cab-df04-455b-ae6a-a919574e39cb None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.681979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 95/188] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:56.682696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65456cab-df04-455b-ae6a-a919574e39cb None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.690677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 94/189] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/role_assignments?user.id=4d94401bc90146c29a9e891bdf7a2ed7&scope.project.id=495160e79d4f4c7e960ab7d794298e67&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:56.690677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.754698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:56.801281 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65456cab-df04-455b-ae6a-a919574e39cb None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.827116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277160f-cd4f-4274-bb8e-4575c58bda97 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.839390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:56.840125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.840407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.840685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.841452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] RBAC: Adding query filter params (name=reader) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:56.841761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:56.843211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:56.854393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 96/190] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:56.855706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:56.866996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99b92d-ab5a-466a-8a98-a7a87ad16d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:56.867834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b92d-ab5a-466a-8a98-a7a87ad16d43 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.868112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b92d-ab5a-466a-8a98-a7a87ad16d43 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.868387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b92d-ab5a-466a-8a98-a7a87ad16d43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.881492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b92d-ab5a-466a-8a98-a7a87ad16d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-fb99b92d-ab5a-466a-8a98-a7a87ad16d43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:39:56.891881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:39:56.892768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:39:56.893550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 95/191] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:39:56.899525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec45595-1b11-422b-b37b-701aebf081e2 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:56.899808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec45595-1b11-422b-b37b-701aebf081e2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:56.900086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec45595-1b11-422b-b37b-701aebf081e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:56.920146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-7ec45595-1b11-422b-b37b-701aebf081e2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:56.921398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7ec45595-1b11-422b-b37b-701aebf081e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:39:56.936107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:56.977145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4c81a-b71b-483d-9d09-460c761a357e None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:56.988085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 96/192] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:39:56.989684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec45595-1b11-422b-b37b-701aebf081e2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.013756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.014475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.014756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.015043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.015807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:57.016114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.017513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.018351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.026976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 97/193] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:57.028037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.091398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec45595-1b11-422b-b37b-701aebf081e2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.131715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dede417-39ce-441c-a881-69c75bc36a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.132182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad73939-60f3-43b3-9e01-d6ca515babb8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.132526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dede417-39ce-441c-a881-69c75bc36a2f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.132809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dede417-39ce-441c-a881-69c75bc36a2f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.133085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dede417-39ce-441c-a881-69c75bc36a2f None admin] PATH_INFO: `/v3/users/6ff07eed450c4e5dba0e1a8d8c3ec7bd` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.134361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dede417-39ce-441c-a881-69c75bc36a2f None admin] RBAC: Authorizing `identity:get_user(user_id=6ff07eed450c4e5dba0e1a8d8c3ec7bd)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.135928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dede417-39ce-441c-a881-69c75bc36a2f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.137607 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 97/194] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:56 2022] GET /identity/v3/users/6ff07eed450c4e5dba0e1a8d8c3ec7bd => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:57.138417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dede417-39ce-441c-a881-69c75bc36a2f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.152856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.166966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.167767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.168051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.168324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.169087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:57.169395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.170817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.178827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 98/195] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:39:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:57.182521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.235481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.244739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dede417-39ce-441c-a881-69c75bc36a2f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.280679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c537-8c8c-4925-90e9-c62451c600a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.281478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c537-8c8c-4925-90e9-c62451c600a8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.281762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c537-8c8c-4925-90e9-c62451c600a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.282069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c537-8c8c-4925-90e9-c62451c600a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.283265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758c537-8c8c-4925-90e9-c62451c600a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.284735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758c537-8c8c-4925-90e9-c62451c600a8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.286419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 98/196] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:39:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:57.287179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c537-8c8c-4925-90e9-c62451c600a8 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.287706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f999a3-bac2-46c7-8c29-130c9d131d95 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.326272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe256d-a05a-41b1-b5cf-bac0087c62be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.327238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe256d-a05a-41b1-b5cf-bac0087c62be None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.327517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe256d-a05a-41b1-b5cf-bac0087c62be None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.327788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe256d-a05a-41b1-b5cf-bac0087c62be None admin] PATH_INFO: `/v3/users/86022e82ef514776b40755664a1f2ccf` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.341399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe256d-a05a-41b1-b5cf-bac0087c62be None admin] RBAC: Authorizing `identity:get_user(user_id=86022e82ef514776b40755664a1f2ccf)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.342909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe256d-a05a-41b1-b5cf-bac0087c62be None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.344783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 99/197] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:57 2022] GET /identity/v3/users/86022e82ef514776b40755664a1f2ccf => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:57.345384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe256d-a05a-41b1-b5cf-bac0087c62be None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.389435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.392691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c537-8c8c-4925-90e9-c62451c600a8 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.430826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f9a08-5980-4da8-9c72-ebdde6653e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.431487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f9a08-5980-4da8-9c72-ebdde6653e74 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.431781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f9a08-5980-4da8-9c72-ebdde6653e74 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.432048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f9a08-5980-4da8-9c72-ebdde6653e74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.456862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe256d-a05a-41b1-b5cf-bac0087c62be None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.463862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.492634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a40a50-8391-4895-9446-e8a8ba81636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.493533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a40a50-8391-4895-9446-e8a8ba81636e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.493845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a40a50-8391-4895-9446-e8a8ba81636e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.494127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a40a50-8391-4895-9446-e8a8ba81636e None admin] PATH_INFO: `/v3/users/86022e82ef514776b40755664a1f2ccf` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.495481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a40a50-8391-4895-9446-e8a8ba81636e None admin] RBAC: Authorizing `identity:get_user(user_id=86022e82ef514776b40755664a1f2ccf)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.497038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a40a50-8391-4895-9446-e8a8ba81636e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.498751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 100/198] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:39:57 2022] GET /identity/v3/users/86022e82ef514776b40755664a1f2ccf => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:57.499491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a40a50-8391-4895-9446-e8a8ba81636e None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.502345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f9a08-5980-4da8-9c72-ebdde6653e74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.504252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f9a08-5980-4da8-9c72-ebdde6653e74 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.611863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a40a50-8391-4895-9446-e8a8ba81636e None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.612646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 99/199] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:57.613510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f9a08-5980-4da8-9c72-ebdde6653e74 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.621729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.649893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32b8c6-ef68-4b48-beeb-bd11a82738a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.650540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b8c6-ef68-4b48-beeb-bd11a82738a1 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.650833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b8c6-ef68-4b48-beeb-bd11a82738a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.651107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32b8c6-ef68-4b48-beeb-bd11a82738a1 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.652141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32b8c6-ef68-4b48-beeb-bd11a82738a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.653678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32b8c6-ef68-4b48-beeb-bd11a82738a1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.673388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 101/200] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:39:57 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:39:57.674252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32b8c6-ef68-4b48-beeb-bd11a82738a1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:57.680845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.724231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f9a08-5980-4da8-9c72-ebdde6653e74 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.765470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9d07c-edbc-4f2f-b54c-53da41bdc812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.766199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d07c-edbc-4f2f-b54c-53da41bdc812 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.766458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d07c-edbc-4f2f-b54c-53da41bdc812 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.766739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d07c-edbc-4f2f-b54c-53da41bdc812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.805481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32b8c6-ef68-4b48-beeb-bd11a82738a1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:57.846935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dff1-d4e4-44c5-b384-770765bc0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:57.847746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dff1-d4e4-44c5-b384-770765bc0303 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:57.848022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dff1-d4e4-44c5-b384-770765bc0303 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:57.848320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dff1-d4e4-44c5-b384-770765bc0303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:57.849846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9d07c-edbc-4f2f-b54c-53da41bdc812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.851492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9d07c-edbc-4f2f-b54c-53da41bdc812 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.890192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.927975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:57.945952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dff1-d4e4-44c5-b384-770765bc0303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:57.948029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dff1-d4e4-44c5-b384-770765bc0303 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:57.977971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 100/201] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:57.978983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9d07c-edbc-4f2f-b54c-53da41bdc812 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:58.087334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 102/202] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:39:58.088209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dff1-d4e4-44c5-b384-770765bc0303 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:58.102568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9d07c-edbc-4f2f-b54c-53da41bdc812 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:58.142939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f9669-8751-4ba2-9153-1cb7de5b98b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:58.144085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9669-8751-4ba2-9153-1cb7de5b98b5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:58.144451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9669-8751-4ba2-9153-1cb7de5b98b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:58.144816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9669-8751-4ba2-9153-1cb7de5b98b5 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:58.146290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f9669-8751-4ba2-9153-1cb7de5b98b5 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:58.147843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f9669-8751-4ba2-9153-1cb7de5b98b5 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:58.155216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 101/203] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:57 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:58.155216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f9669-8751-4ba2-9153-1cb7de5b98b5 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:58.172706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:58.189948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:58.217466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dff1-d4e4-44c5-b384-770765bc0303 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:58.255910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3460cf-1d9e-4e41-8b9c-97d692e4b649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:58.257097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3460cf-1d9e-4e41-8b9c-97d692e4b649 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:58.257616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3460cf-1d9e-4e41-8b9c-97d692e4b649 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:58.258120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3460cf-1d9e-4e41-8b9c-97d692e4b649 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:58.259545 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3460cf-1d9e-4e41-8b9c-97d692e4b649 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:58.261383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3460cf-1d9e-4e41-8b9c-97d692e4b649 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:58.268841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 103/204] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:58 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:58.270133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3460cf-1d9e-4e41-8b9c-97d692e4b649 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:58.274236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f9669-8751-4ba2-9153-1cb7de5b98b5 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:58.312594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc3cc2-cc34-4963-99d3-54cc6d290af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:58.313626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc3cc2-cc34-4963-99d3-54cc6d290af7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:58.314007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc3cc2-cc34-4963-99d3-54cc6d290af7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:58.314351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc3cc2-cc34-4963-99d3-54cc6d290af7 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:58.315900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc3cc2-cc34-4963-99d3-54cc6d290af7 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:58.317454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc3cc2-cc34-4963-99d3-54cc6d290af7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:58.319324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 102/205] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:39:58 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:58.320349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc3cc2-cc34-4963-99d3-54cc6d290af7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:58.383512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3460cf-1d9e-4e41-8b9c-97d692e4b649 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:58.420718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:58.421767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:58.422382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:58.423006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:58.423354 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:58.425032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] RBAC: Adding query filter params (role.id=24a12434807d480fafb72632fc2d6756, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62, user.id=6ff07eed450c4e5dba0e1a8d8c3ec7bd) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:58.425539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:58.426098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc3cc2-cc34-4963-99d3-54cc6d290af7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:58.428138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:39:58.428679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:58.437026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 104/206] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:58 2022] GET /identity/v3/role_assignments?user.id=6ff07eed450c4e5dba0e1a8d8c3ec7bd&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=24a12434807d480fafb72632fc2d6756 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:58.438414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:39:58.464166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28e0f3-fe8e-42b5-922c-21789e2fdc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:58.465245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28e0f3-fe8e-42b5-922c-21789e2fdc6d None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:58.465590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28e0f3-fe8e-42b5-922c-21789e2fdc6d None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:58.465909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28e0f3-fe8e-42b5-922c-21789e2fdc6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:58.466806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28e0f3-fe8e-42b5-922c-21789e2fdc6d None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=20a1163a83aa4705a89b4635d4e037b4, user.id=86022e82ef514776b40755664a1f2ccf) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:58.467124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28e0f3-fe8e-42b5-922c-21789e2fdc6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:58.468834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28e0f3-fe8e-42b5-922c-21789e2fdc6d None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:58.475434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 103/207] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:58 2022] GET /identity/v3/role_assignments?user.id=86022e82ef514776b40755664a1f2ccf&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:58.559347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7b85d-6a8e-40ad-9c5b-b1e6b3dea131 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:39:58.599645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1065b4-c4fb-42d8-9842-b0bee0bc7a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:39:58.600721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1065b4-c4fb-42d8-9842-b0bee0bc7a13 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:58.601193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1065b4-c4fb-42d8-9842-b0bee0bc7a13 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:58.601683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1065b4-c4fb-42d8-9842-b0bee0bc7a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:58.602817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1065b4-c4fb-42d8-9842-b0bee0bc7a13 None admin] RBAC: Adding query filter params (role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=20a1163a83aa4705a89b4635d4e037b4, user.id=86022e82ef514776b40755664a1f2ccf) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:39:58.603368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1065b4-c4fb-42d8-9842-b0bee0bc7a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:39:58.605189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1065b4-c4fb-42d8-9842-b0bee0bc7a13 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:39:58.612096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 105/208] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:39:58 2022] GET /identity/v3/role_assignments?user.id=86022e82ef514776b40755664a1f2ccf&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:39:59.958882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d554-75d4-4b51-85f8-1883e405985a None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:59.959405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d554-75d4-4b51-85f8-1883e405985a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:59.959405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d554-75d4-4b51-85f8-1883e405985a None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:59.961155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 104/209] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:39:59.966446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786df83-522b-489e-82bd-d498fcf25482 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:39:59.966886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786df83-522b-489e-82bd-d498fcf25482 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:39:59.967056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786df83-522b-489e-82bd-d498fcf25482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:39:59.984273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2786df83-522b-489e-82bd-d498fcf25482 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:39:59.985472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2786df83-522b-489e-82bd-d498fcf25482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:00.011898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa0a7-5fd2-4564-bdcf-71295fb85603 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.012187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa0a7-5fd2-4564-bdcf-71295fb85603 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.012465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa0a7-5fd2-4564-bdcf-71295fb85603 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.014121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 105/210] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:00.019210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3a439-5bf2-4159-ae8e-e7e59d5796e0 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.019729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3a439-5bf2-4159-ae8e-e7e59d5796e0 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.020215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3a439-5bf2-4159-ae8e-e7e59d5796e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.039597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-eaf3a439-5bf2-4159-ae8e-e7e59d5796e0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:00.041136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-eaf3a439-5bf2-4159-ae8e-e7e59d5796e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:00.046117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:00.061812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 106/211] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:00.083443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786df83-522b-489e-82bd-d498fcf25482 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:00.122942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 106/212] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:00.147693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3a439-5bf2-4159-ae8e-e7e59d5796e0 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:00.205288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786df83-522b-489e-82bd-d498fcf25482 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:00.246501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:00.253006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e251c-b404-415c-9ab3-2b3273cc3534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:00.254169 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e251c-b404-415c-9ab3-2b3273cc3534 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.254457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e251c-b404-415c-9ab3-2b3273cc3534 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.255063 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e251c-b404-415c-9ab3-2b3273cc3534 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.256952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3a439-5bf2-4159-ae8e-e7e59d5796e0 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:00.270079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e251c-b404-415c-9ab3-2b3273cc3534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-558e251c-b404-415c-9ab3-2b3273cc3534 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:00.279421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:00.280275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:00.281058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:00.283654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 107/213] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:00.290954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e251c-b404-415c-9ab3-2b3273cc3534 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:00.301571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e36ca6-3f28-45eb-bb5a-30fdcb2952a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:00.302772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e36ca6-3f28-45eb-bb5a-30fdcb2952a8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.303246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e36ca6-3f28-45eb-bb5a-30fdcb2952a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.303757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e36ca6-3f28-45eb-bb5a-30fdcb2952a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.305161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e36ca6-3f28-45eb-bb5a-30fdcb2952a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:00.306894 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e36ca6-3f28-45eb-bb5a-30fdcb2952a8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:00.313707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 107/214] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:00.314893 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e36ca6-3f28-45eb-bb5a-30fdcb2952a8 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:00.355237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:00.417898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e251c-b404-415c-9ab3-2b3273cc3534 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:00.437075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e36ca6-3f28-45eb-bb5a-30fdcb2952a8 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:00.455644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:00.456713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.457187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.457802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.458853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] RBAC: Adding query filter params (name=member) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:00.459374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:00.461053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:00.470859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 108/215] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:00.476755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:00.481541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a27755-ca0d-4876-b468-7454de5e84b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:00.482539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27755-ca0d-4876-b468-7454de5e84b1 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.483016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27755-ca0d-4876-b468-7454de5e84b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.483486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a27755-ca0d-4876-b468-7454de5e84b1 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.484418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a27755-ca0d-4876-b468-7454de5e84b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:00.486269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a27755-ca0d-4876-b468-7454de5e84b1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:00.520675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 108/216] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:40:00 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:00.595667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf66111-7780-46c5-89f0-7a2846e7abfa None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:00.611054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:00.623716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d96e-c769-42fc-b503-b8db5f9da000 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.624071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d96e-c769-42fc-b503-b8db5f9da000 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.624279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0d96e-c769-42fc-b503-b8db5f9da000 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.626017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 109/217] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:40:00.633379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936f3a7-6b56-4bff-922c-edc8c029c73e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.633815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936f3a7-6b56-4bff-922c-edc8c029c73e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.634215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936f3a7-6b56-4bff-922c-edc8c029c73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.637030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcba1f-c535-4976-90cc-b651af8dc7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:00.637976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcba1f-c535-4976-90cc-b651af8dc7eb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.638259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcba1f-c535-4976-90cc-b651af8dc7eb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.638530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcba1f-c535-4976-90cc-b651af8dc7eb None admin] PATH_INFO: `/v3/users/0dd7ab851f154030963985c6a4f7d3e4` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.654788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcba1f-c535-4976-90cc-b651af8dc7eb None admin] RBAC: Authorizing `identity:get_user(user_id=0dd7ab851f154030963985c6a4f7d3e4)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:00.655159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1936f3a7-6b56-4bff-922c-edc8c029c73e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:00.656282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcba1f-c535-4976-90cc-b651af8dc7eb None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:00.656383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1936f3a7-6b56-4bff-922c-edc8c029c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:00.658460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 109/218] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:00 2022] GET /identity/v3/users/0dd7ab851f154030963985c6a4f7d3e4 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:00.663948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcba1f-c535-4976-90cc-b651af8dc7eb None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:00.679657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:00.757086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 110/219] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:00.758619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020c94a-cad7-4247-93e0-67ab4ee835b2 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.758915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020c94a-cad7-4247-93e0-67ab4ee835b2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.759193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020c94a-cad7-4247-93e0-67ab4ee835b2 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.760773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 111/220] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:00.766564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892d977-3b68-46c5-847d-cf3edbbab5a3 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.766564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892d977-3b68-46c5-847d-cf3edbbab5a3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.766776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892d977-3b68-46c5-847d-cf3edbbab5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.782743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcba1f-c535-4976-90cc-b651af8dc7eb None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:00.786906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5892d977-3b68-46c5-847d-cf3edbbab5a3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:00.788322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5892d977-3b68-46c5-847d-cf3edbbab5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:00.822456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624c4f1-5f0a-40a5-863d-1736cff13bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:00.823517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624c4f1-5f0a-40a5-863d-1736cff13bbe None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:00.823959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624c4f1-5f0a-40a5-863d-1736cff13bbe None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:00.824243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624c4f1-5f0a-40a5-863d-1736cff13bbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:00.877510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 112/221] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:00.878538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892d977-3b68-46c5-847d-cf3edbbab5a3 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:00.880627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:00.914107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624c4f1-5f0a-40a5-863d-1736cff13bbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:00.916391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624c4f1-5f0a-40a5-863d-1736cff13bbe None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:00.998060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:01.028889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892d977-3b68-46c5-847d-cf3edbbab5a3 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.047679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 110/222] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 15:40:01.048479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624c4f1-5f0a-40a5-863d-1736cff13bbe None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.073527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b883fd-9ad4-4a9c-a5fc-4d30e8da9348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.075104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b883fd-9ad4-4a9c-a5fc-4d30e8da9348 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.075390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b883fd-9ad4-4a9c-a5fc-4d30e8da9348 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.075668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b883fd-9ad4-4a9c-a5fc-4d30e8da9348 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.090667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b883fd-9ad4-4a9c-a5fc-4d30e8da9348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-20b883fd-9ad4-4a9c-a5fc-4d30e8da9348 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:01.100633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:01.101538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:01.102317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:01.103067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 113/223] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:01.104182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b883fd-9ad4-4a9c-a5fc-4d30e8da9348 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.136259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:01.179634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624c4f1-5f0a-40a5-863d-1736cff13bbe None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.226880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63434897-771a-49ed-b9fa-a83c1efd43ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.228047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63434897-771a-49ed-b9fa-a83c1efd43ae None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.228568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63434897-771a-49ed-b9fa-a83c1efd43ae None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.228847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63434897-771a-49ed-b9fa-a83c1efd43ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.235057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b883fd-9ad4-4a9c-a5fc-4d30e8da9348 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.258461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63434897-771a-49ed-b9fa-a83c1efd43ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-63434897-771a-49ed-b9fa-a83c1efd43ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:01.262497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:01.263575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:01.264571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:01.269701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 111/224] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:01.270640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63434897-771a-49ed-b9fa-a83c1efd43ae None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.277228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdcc3b-92d2-414d-b734-54071ca75d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.278261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdcc3b-92d2-414d-b734-54071ca75d03 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.278644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdcc3b-92d2-414d-b734-54071ca75d03 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.279019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdcc3b-92d2-414d-b734-54071ca75d03 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.280369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdcc3b-92d2-414d-b734-54071ca75d03 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:01.281993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdcc3b-92d2-414d-b734-54071ca75d03 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:01.284146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 114/225] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:01.285092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdcc3b-92d2-414d-b734-54071ca75d03 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.301002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:01.405143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63434897-771a-49ed-b9fa-a83c1efd43ae None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.420548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdcc3b-92d2-414d-b734-54071ca75d03 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.437434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:01.444743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.445710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.446154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.446645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.447509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:01.447924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:01.449476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:01.459178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 112/226] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:01.460193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.467646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.468602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.469027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.469445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.470424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:01.471464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:01.473515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:01.485022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 115/227] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:01.490697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.569743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:01.598778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e6451-7995-4ffb-a758-42ba21deb337 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.606269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7bae-ab9a-48c3-b7e4-d0a937f0540b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.645078 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69993dc3-dc9e-4c09-a7a7-49414717c683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.646001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69993dc3-dc9e-4c09-a7a7-49414717c683 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.646282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69993dc3-dc9e-4c09-a7a7-49414717c683 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.646556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69993dc3-dc9e-4c09-a7a7-49414717c683 None admin] PATH_INFO: `/v3/users/86022e82ef514776b40755664a1f2ccf` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.647762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69993dc3-dc9e-4c09-a7a7-49414717c683 None admin] RBAC: Authorizing `identity:get_user(user_id=86022e82ef514776b40755664a1f2ccf)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:01.649068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.649598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69993dc3-dc9e-4c09-a7a7-49414717c683 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:01.650185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.650654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.651129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.651363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 116/228] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/users/86022e82ef514776b40755664a1f2ccf => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:01.656157 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69993dc3-dc9e-4c09-a7a7-49414717c683 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.656280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] RBAC: Adding query filter params (role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=20a1163a83aa4705a89b4635d4e037b4, user.id=0dd7ab851f154030963985c6a4f7d3e4) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:01.656280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:01.656280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:01.662928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 113/229] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/role_assignments?user.id=0dd7ab851f154030963985c6a4f7d3e4&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:01.664037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.726382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:01.778855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0460-16e2-449d-91d2-81d3242a73dd None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.814076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69993dc3-dc9e-4c09-a7a7-49414717c683 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:01.846414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09655b2d-801c-4df8-bb3b-34b05f1544e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.848023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09655b2d-801c-4df8-bb3b-34b05f1544e5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.848324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09655b2d-801c-4df8-bb3b-34b05f1544e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.848604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09655b2d-801c-4df8-bb3b-34b05f1544e5 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.850755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09655b2d-801c-4df8-bb3b-34b05f1544e5 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:01.852259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09655b2d-801c-4df8-bb3b-34b05f1544e5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:01.854374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 114/230] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:01.859584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f8a2c-8058-4d69-adcd-cb3cd5403f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:01.860498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8a2c-8058-4d69-adcd-cb3cd5403f0e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:01.860782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8a2c-8058-4d69-adcd-cb3cd5403f0e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:01.861056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8a2c-8058-4d69-adcd-cb3cd5403f0e None admin] PATH_INFO: `/v3/users/86022e82ef514776b40755664a1f2ccf` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:01.862580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09655b2d-801c-4df8-bb3b-34b05f1544e5 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.863668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f8a2c-8058-4d69-adcd-cb3cd5403f0e None admin] RBAC: Authorizing `identity:get_user(user_id=86022e82ef514776b40755664a1f2ccf)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:01.865178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f8a2c-8058-4d69-adcd-cb3cd5403f0e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:01.867001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 117/231] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/users/86022e82ef514776b40755664a1f2ccf => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:01.875280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f8a2c-8058-4d69-adcd-cb3cd5403f0e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:01.885926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:02.012546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f8a2c-8058-4d69-adcd-cb3cd5403f0e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:02.051588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b9865-7a76-46da-8fde-be9387c1c5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:02.052478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b9865-7a76-46da-8fde-be9387c1c5fa None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:02.052761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b9865-7a76-46da-8fde-be9387c1c5fa None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:02.053046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b9865-7a76-46da-8fde-be9387c1c5fa None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:02.054268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b9865-7a76-46da-8fde-be9387c1c5fa None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:02.057978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7b9865-7a76-46da-8fde-be9387c1c5fa None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:02.058197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 118/232] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:01 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:02.064639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b9865-7a76-46da-8fde-be9387c1c5fa None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:02.080058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09655b2d-801c-4df8-bb3b-34b05f1544e5 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:02.126366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:02.137924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0011e2-2815-4adc-a025-29c61b1adc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:02.139329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0011e2-2815-4adc-a025-29c61b1adc87 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:02.139933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0011e2-2815-4adc-a025-29c61b1adc87 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:02.140410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0011e2-2815-4adc-a025-29c61b1adc87 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4/users/86022e82ef514776b40755664a1f2ccf/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:02.142677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0011e2-2815-4adc-a025-29c61b1adc87 None admin] RBAC: Authorizing `identity:create_grant(project_id=20a1163a83aa4705a89b4635d4e037b4, user_id=86022e82ef514776b40755664a1f2ccf, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:02.144351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0011e2-2815-4adc-a025-29c61b1adc87 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:02.161439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 115/233] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:01 2022] PUT /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4/users/86022e82ef514776b40755664a1f2ccf/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:02.176235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:02.212770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b9865-7a76-46da-8fde-be9387c1c5fa None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:02.264081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023ac14-6fdd-4fe3-8e78-6015150acbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:02.265039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023ac14-6fdd-4fe3-8e78-6015150acbfe None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:02.265323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023ac14-6fdd-4fe3-8e78-6015150acbfe None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:02.265600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023ac14-6fdd-4fe3-8e78-6015150acbfe None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4/users/86022e82ef514776b40755664a1f2ccf/roles/a505b2a721044e6ea8312ef6ffdc1c9f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:02.268210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023ac14-6fdd-4fe3-8e78-6015150acbfe None admin] RBAC: Authorizing `identity:create_grant(project_id=20a1163a83aa4705a89b4635d4e037b4, user_id=86022e82ef514776b40755664a1f2ccf, role_id=a505b2a721044e6ea8312ef6ffdc1c9f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:02.269821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023ac14-6fdd-4fe3-8e78-6015150acbfe None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:02.289759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 119/234] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:02 2022] PUT /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4/users/86022e82ef514776b40755664a1f2ccf/roles/a505b2a721044e6ea8312ef6ffdc1c9f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:40:02.961423 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b742e14-45b1-4420-bc9d-4a6181e5b13f None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:02.962098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b742e14-45b1-4420-bc9d-4a6181e5b13f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:02.962582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b742e14-45b1-4420-bc9d-4a6181e5b13f None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:02.964579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 116/235] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:02.970263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca25b-4bb4-4543-8b4e-9a0fc9e64d60 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:02.970824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca25b-4bb4-4543-8b4e-9a0fc9e64d60 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:02.971301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca25b-4bb4-4543-8b4e-9a0fc9e64d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:03.001478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-50cca25b-4bb4-4543-8b4e-9a0fc9e64d60 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:03.003092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-50cca25b-4bb4-4543-8b4e-9a0fc9e64d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:03.084554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a96c1-0cbc-4edb-8170-07cc1e2bab56 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:03.085518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a96c1-0cbc-4edb-8170-07cc1e2bab56 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:03.086081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a96c1-0cbc-4edb-8170-07cc1e2bab56 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:03.087885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 117/236] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:03.090553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 120/237] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:03.105346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf927-e4c1-478d-814a-14ac1ebe60bd None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:03.105983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf927-e4c1-478d-814a-14ac1ebe60bd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:03.106437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedf927-e4c1-478d-814a-14ac1ebe60bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:03.137097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-afedf927-e4c1-478d-814a-14ac1ebe60bd None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:03.137487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cca25b-4bb4-4543-8b4e-9a0fc9e64d60 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:03.138725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-afedf927-e4c1-478d-814a-14ac1ebe60bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:03.253123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 118/238] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:03.260485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:03.316821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedf927-e4c1-478d-814a-14ac1ebe60bd None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:03.318289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cca25b-4bb4-4543-8b4e-9a0fc9e64d60 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:03.419053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcb4aa-e943-41d3-a8d7-80d492d266e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:03.419890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcb4aa-e943-41d3-a8d7-80d492d266e4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:03.420177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcb4aa-e943-41d3-a8d7-80d492d266e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:03.420457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcb4aa-e943-41d3-a8d7-80d492d266e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:03.421642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcb4aa-e943-41d3-a8d7-80d492d266e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:03.423162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcb4aa-e943-41d3-a8d7-80d492d266e4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:03.425113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 121/239] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:03.430145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcb4aa-e943-41d3-a8d7-80d492d266e4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:03.440238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedf927-e4c1-478d-814a-14ac1ebe60bd None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:03.456913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:03.479925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fbf30-7aaa-47b2-a0d7-0879e829a5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:03.480896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fbf30-7aaa-47b2-a0d7-0879e829a5c9 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:03.481729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fbf30-7aaa-47b2-a0d7-0879e829a5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:03.482120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fbf30-7aaa-47b2-a0d7-0879e829a5c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:03.494397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fbf30-7aaa-47b2-a0d7-0879e829a5c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-303fbf30-7aaa-47b2-a0d7-0879e829a5c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:03.503580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:03.504459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:03.505259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:03.505887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 119/240] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:03.511808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fbf30-7aaa-47b2-a0d7-0879e829a5c9 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:03.624979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fbf30-7aaa-47b2-a0d7-0879e829a5c9 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:03.673628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:03.675679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:03.675964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:03.676238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:03.677004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] RBAC: Adding query filter params (name=reader) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:03.677313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:03.678750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:03.696822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 120/241] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:03.706268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:03.717390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcb4aa-e943-41d3-a8d7-80d492d266e4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:03.778248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:03.782547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:03.793166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce82e4-bb0c-4f3a-844b-9ef677a16ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:03.794002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce82e4-bb0c-4f3a-844b-9ef677a16ee6 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:03.794291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce82e4-bb0c-4f3a-844b-9ef677a16ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:03.794574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce82e4-bb0c-4f3a-844b-9ef677a16ee6 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:03.795329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce82e4-bb0c-4f3a-844b-9ef677a16ee6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:03.796987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce82e4-bb0c-4f3a-844b-9ef677a16ee6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:03.835954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 122/242] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:40:03 2022] POST /identity/v3/users => generated 293 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:40:03.871472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920756-6629-425b-ad5d-4f6abc5f5cdc None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:03.910283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5777f5-6ce7-474a-b250-c517e34f8fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:03.911208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5777f5-6ce7-474a-b250-c517e34f8fa0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:03.911489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5777f5-6ce7-474a-b250-c517e34f8fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:03.911763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5777f5-6ce7-474a-b250-c517e34f8fa0 None admin] PATH_INFO: `/v3/users/823c2d577e864ab38f954db574c67187` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:03.927620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5777f5-6ce7-474a-b250-c517e34f8fa0 None admin] RBAC: Authorizing `identity:get_user(user_id=823c2d577e864ab38f954db574c67187)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:03.929108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5777f5-6ce7-474a-b250-c517e34f8fa0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:03.931220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 121/243] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:03 2022] GET /identity/v3/users/823c2d577e864ab38f954db574c67187 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:03.936950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce82e4-bb0c-4f3a-844b-9ef677a16ee6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:04.099473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce82e4-bb0c-4f3a-844b-9ef677a16ee6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:04.136857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243e5f-aff3-4e0c-97d7-3d663f802e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:04.137541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243e5f-aff3-4e0c-97d7-3d663f802e77 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.137948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243e5f-aff3-4e0c-97d7-3d663f802e77 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.138226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243e5f-aff3-4e0c-97d7-3d663f802e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.210219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:04.223649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b243e5f-aff3-4e0c-97d7-3d663f802e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:04.223649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b243e5f-aff3-4e0c-97d7-3d663f802e77 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:04.356358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 123/244] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:04.362197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5777f5-6ce7-474a-b250-c517e34f8fa0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:04.427220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99024e31-0b1c-455b-aa05-8ca10673a7a6 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.427508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99024e31-0b1c-455b-aa05-8ca10673a7a6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.427785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99024e31-0b1c-455b-aa05-8ca10673a7a6 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.429387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 124/245] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:04.435317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ac4dc-6e78-4603-a136-87c3b826fc0b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.435716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ac4dc-6e78-4603-a136-87c3b826fc0b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.436059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ac4dc-6e78-4603-a136-87c3b826fc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.456444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5c8ac4dc-6e78-4603-a136-87c3b826fc0b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:04.459079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5c8ac4dc-6e78-4603-a136-87c3b826fc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:04.528096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:04.540051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5777f5-6ce7-474a-b250-c517e34f8fa0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:04.544424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 125/246] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:04.568534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ac4dc-6e78-4603-a136-87c3b826fc0b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:04.581038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad0c71-31df-40ba-8055-6b3ee42f81ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:04.582385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad0c71-31df-40ba-8055-6b3ee42f81ba None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.582871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad0c71-31df-40ba-8055-6b3ee42f81ba None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.583335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad0c71-31df-40ba-8055-6b3ee42f81ba None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.584801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad0c71-31df-40ba-8055-6b3ee42f81ba None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:04.590837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:04.591540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad0c71-31df-40ba-8055-6b3ee42f81ba None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:04.593764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 122/247] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:04 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:04.610210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad0c71-31df-40ba-8055-6b3ee42f81ba None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:04.693402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ac4dc-6e78-4603-a136-87c3b826fc0b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:04.738925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f21567-3d31-46b6-82b8-e7081a0e8cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:04.739764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f21567-3d31-46b6-82b8-e7081a0e8cac None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.740046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f21567-3d31-46b6-82b8-e7081a0e8cac None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.740322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f21567-3d31-46b6-82b8-e7081a0e8cac None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.753688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f21567-3d31-46b6-82b8-e7081a0e8cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-84f21567-3d31-46b6-82b8-e7081a0e8cac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:04.763254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:04.764196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:04.765472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 126/248] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:04.771176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f21567-3d31-46b6-82b8-e7081a0e8cac None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:04.813985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad0c71-31df-40ba-8055-6b3ee42f81ba None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:04.858347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:04.860397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7232f-ebbf-4301-b030-a58e24463242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:04.861255 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7232f-ebbf-4301-b030-a58e24463242 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.861671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7232f-ebbf-4301-b030-a58e24463242 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.862111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7232f-ebbf-4301-b030-a58e24463242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.863150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7232f-ebbf-4301-b030-a58e24463242 None admin] RBAC: Adding query filter params (role.id=24a12434807d480fafb72632fc2d6756, scope.project.id=20a1163a83aa4705a89b4635d4e037b4, user.id=823c2d577e864ab38f954db574c67187) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:04.867398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7232f-ebbf-4301-b030-a58e24463242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:04.867398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7232f-ebbf-4301-b030-a58e24463242 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:04.872441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 123/249] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:04 2022] GET /identity/v3/role_assignments?user.id=823c2d577e864ab38f954db574c67187&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=24a12434807d480fafb72632fc2d6756 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:04.874100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818da0e-8383-4dc7-8ced-00ed4c268792 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.874552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818da0e-8383-4dc7-8ced-00ed4c268792 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.874963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818da0e-8383-4dc7-8ced-00ed4c268792 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.880696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 124/250] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:04.882233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52c53e-5dc9-4064-83bb-c45eb0f681e5 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.882696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52c53e-5dc9-4064-83bb-c45eb0f681e5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.883110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52c53e-5dc9-4064-83bb-c45eb0f681e5 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.884584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 125/251] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:04.888806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51879285-a5ca-4c45-9e9e-d9e59ca7f37f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:04.892366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51879285-a5ca-4c45-9e9e-d9e59ca7f37f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:04.893096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51879285-a5ca-4c45-9e9e-d9e59ca7f37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:04.905998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f21567-3d31-46b6-82b8-e7081a0e8cac None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:04.953386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-51879285-a5ca-4c45-9e9e-d9e59ca7f37f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:04.955813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-51879285-a5ca-4c45-9e9e-d9e59ca7f37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:04.998583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:05.051600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca277a-fc49-498a-b614-7dab29bd8016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:05.052336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca277a-fc49-498a-b614-7dab29bd8016 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:05.052621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca277a-fc49-498a-b614-7dab29bd8016 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:05.052901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca277a-fc49-498a-b614-7dab29bd8016 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:05.053672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca277a-fc49-498a-b614-7dab29bd8016 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:05.054011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca277a-fc49-498a-b614-7dab29bd8016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:05.055430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca277a-fc49-498a-b614-7dab29bd8016 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:05.066506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 127/252] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:05.067850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c4c2-4844-41fe-9fbb-f747b2084c56 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:05.068112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c4c2-4844-41fe-9fbb-f747b2084c56 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:05.068369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c4c2-4844-41fe-9fbb-f747b2084c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:05.089789 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3bc9c4c2-4844-41fe-9fbb-f747b2084c56 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:05.091026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3bc9c4c2-4844-41fe-9fbb-f747b2084c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:05.105085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 126/253] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:05.107240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51879285-a5ca-4c45-9e9e-d9e59ca7f37f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:05.161445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 128/254] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:05.180256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c4c2-4844-41fe-9fbb-f747b2084c56 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:05.318473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51879285-a5ca-4c45-9e9e-d9e59ca7f37f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:05.343177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:05.364531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f73d1-4475-4958-94c6-e7f473093c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:05.365459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f73d1-4475-4958-94c6-e7f473093c6a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:05.365749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f73d1-4475-4958-94c6-e7f473093c6a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:05.366012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f73d1-4475-4958-94c6-e7f473093c6a None admin] PATH_INFO: `/v3/users/0dd7ab851f154030963985c6a4f7d3e4` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:05.368759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f73d1-4475-4958-94c6-e7f473093c6a None admin] RBAC: Authorizing `identity:get_user(user_id=0dd7ab851f154030963985c6a4f7d3e4)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:05.370395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f73d1-4475-4958-94c6-e7f473093c6a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:05.373206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 127/255] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:05 2022] GET /identity/v3/users/0dd7ab851f154030963985c6a4f7d3e4 => generated 297 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:05.374940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f73d1-4475-4958-94c6-e7f473093c6a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:05.405270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c4c2-4844-41fe-9fbb-f747b2084c56 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:05.426286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:05.468632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e24af-ddcd-4124-a099-0d50ee9b31d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:05.469458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e24af-ddcd-4124-a099-0d50ee9b31d4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:05.469741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e24af-ddcd-4124-a099-0d50ee9b31d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:05.470043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e24af-ddcd-4124-a099-0d50ee9b31d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:05.491087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e24af-ddcd-4124-a099-0d50ee9b31d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-651e24af-ddcd-4124-a099-0d50ee9b31d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:05.498906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:05.499771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:05.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 129/256] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:05.501175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e24af-ddcd-4124-a099-0d50ee9b31d4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:05.556856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f73d1-4475-4958-94c6-e7f473093c6a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:05.602573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ea8a-3d37-4ae3-b8d8-7b2424e22982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:05.603401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ea8a-3d37-4ae3-b8d8-7b2424e22982 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:05.603686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ea8a-3d37-4ae3-b8d8-7b2424e22982 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:05.603963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ea8a-3d37-4ae3-b8d8-7b2424e22982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:05.626667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633ea8a-3d37-4ae3-b8d8-7b2424e22982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-5633ea8a-3d37-4ae3-b8d8-7b2424e22982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:05.634873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:05.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:05.637259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:05.646231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 128/257] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:05.647161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ea8a-3d37-4ae3-b8d8-7b2424e22982 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:05.664781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e24af-ddcd-4124-a099-0d50ee9b31d4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:05.694259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:05.709799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec153c-44a7-437f-ace9-7505bc32370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:05.709799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec153c-44a7-437f-ace9-7505bc32370a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:05.709799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec153c-44a7-437f-ace9-7505bc32370a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:05.709799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec153c-44a7-437f-ace9-7505bc32370a None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:05.711017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec153c-44a7-437f-ace9-7505bc32370a None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:05.712511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec153c-44a7-437f-ace9-7505bc32370a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:05.714479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 130/258] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:05 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:05.719655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec153c-44a7-437f-ace9-7505bc32370a None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:05.789833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:05.821374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ea8a-3d37-4ae3-b8d8-7b2424e22982 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:05.893021 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:05.893712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:05.894020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:05.894300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:05.895077 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:05.895388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:05.896799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:05.910537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 129/259] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:05.911184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:05.928844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec153c-44a7-437f-ace9-7505bc32370a None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:05.992604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:05.993297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:05.993582 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:05.993896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:05.994665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:05.995009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:05.996455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:06.015060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 131/260] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:06.016048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:06.093858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491194a7-90ab-45d3-a8fb-966b7e7b1fa3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:06.133727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1401941-1115-49c7-a3e9-2dd8c935f224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:06.134728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1401941-1115-49c7-a3e9-2dd8c935f224 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:06.135017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1401941-1115-49c7-a3e9-2dd8c935f224 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:06.135295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1401941-1115-49c7-a3e9-2dd8c935f224 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4/users/0dd7ab851f154030963985c6a4f7d3e4/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:06.137438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1401941-1115-49c7-a3e9-2dd8c935f224 None admin] RBAC: Authorizing `identity:create_grant(project_id=20a1163a83aa4705a89b4635d4e037b4, user_id=0dd7ab851f154030963985c6a4f7d3e4, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:06.140145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1401941-1115-49c7-a3e9-2dd8c935f224 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:06.164667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 130/261] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:05 2022] PUT /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4/users/0dd7ab851f154030963985c6a4f7d3e4/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:06.165586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1401941-1115-49c7-a3e9-2dd8c935f224 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:06.166824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:06.187327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:06.217826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae20f8-de53-41da-a6a1-0903cceeee26 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:06.277055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d9987-cc7b-4ac7-b402-8c4ee4070c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:06.277842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d9987-cc7b-4ac7-b402-8c4ee4070c41 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:06.278138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d9987-cc7b-4ac7-b402-8c4ee4070c41 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:06.278427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d9987-cc7b-4ac7-b402-8c4ee4070c41 None admin] PATH_INFO: `/v3/users/86022e82ef514776b40755664a1f2ccf` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:06.282667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d9987-cc7b-4ac7-b402-8c4ee4070c41 None admin] RBAC: Authorizing `identity:get_user(user_id=86022e82ef514776b40755664a1f2ccf)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:06.284208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d9987-cc7b-4ac7-b402-8c4ee4070c41 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:06.286101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 132/262] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:06 2022] GET /identity/v3/users/86022e82ef514776b40755664a1f2ccf => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:06.291770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d9987-cc7b-4ac7-b402-8c4ee4070c41 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:06.365197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1401941-1115-49c7-a3e9-2dd8c935f224 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:06.466726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1766e-86e5-45c6-b7dd-7ddd846de06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:06.467646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1766e-86e5-45c6-b7dd-7ddd846de06d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:06.467929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1766e-86e5-45c6-b7dd-7ddd846de06d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:06.468208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1766e-86e5-45c6-b7dd-7ddd846de06d None admin] PATH_INFO: `/v3/users/86022e82ef514776b40755664a1f2ccf` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:06.475892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1766e-86e5-45c6-b7dd-7ddd846de06d None admin] RBAC: Authorizing `identity:get_user(user_id=86022e82ef514776b40755664a1f2ccf)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:06.477925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1766e-86e5-45c6-b7dd-7ddd846de06d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:06.480719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 131/263] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:06 2022] GET /identity/v3/users/86022e82ef514776b40755664a1f2ccf => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:06.486523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1766e-86e5-45c6-b7dd-7ddd846de06d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:06.507264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d9987-cc7b-4ac7-b402-8c4ee4070c41 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:06.545290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:06.559237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c8ed7-cde3-47c4-b4f7-562519f9f8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:06.560115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c8ed7-cde3-47c4-b4f7-562519f9f8a2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:06.560404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c8ed7-cde3-47c4-b4f7-562519f9f8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:06.560681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c8ed7-cde3-47c4-b4f7-562519f9f8a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:06.600184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:06.652686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c8ed7-cde3-47c4-b4f7-562519f9f8a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:06.654380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c8ed7-cde3-47c4-b4f7-562519f9f8a2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:06.656801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1766e-86e5-45c6-b7dd-7ddd846de06d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:06.754533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e7598-b69f-44e6-bcc4-fe7b36fd968d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:06.755217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e7598-b69f-44e6-bcc4-fe7b36fd968d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:06.755504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e7598-b69f-44e6-bcc4-fe7b36fd968d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:06.755782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e7598-b69f-44e6-bcc4-fe7b36fd968d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:06.922316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 133/264] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:06.927835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e7598-b69f-44e6-bcc4-fe7b36fd968d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:06.927990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c8ed7-cde3-47c4-b4f7-562519f9f8a2 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:06.929352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561e7598-b69f-44e6-bcc4-fe7b36fd968d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:06.999986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:07.055953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:07.092427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 132/265] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 15:40:07.094074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129b285-9516-4499-a223-a6ecad2500c7 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.095671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129b285-9516-4499-a223-a6ecad2500c7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.095952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129b285-9516-4499-a223-a6ecad2500c7 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.097316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 133/266] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:07.098047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129b285-9516-4499-a223-a6ecad2500c7 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:07.117145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c8ed7-cde3-47c4-b4f7-562519f9f8a2 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:07.162595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6b13c-0db0-46aa-9ac6-4bf8830c5669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:07.163502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6b13c-0db0-46aa-9ac6-4bf8830c5669 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.163782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6b13c-0db0-46aa-9ac6-4bf8830c5669 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.164058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6b13c-0db0-46aa-9ac6-4bf8830c5669 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.165263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6b13c-0db0-46aa-9ac6-4bf8830c5669 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:07.167243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6b13c-0db0-46aa-9ac6-4bf8830c5669 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:07.169516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 134/267] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:06 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:07.170806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09763-0611-4be7-b559-0db9dad31c04 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.171090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09763-0611-4be7-b559-0db9dad31c04 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.171368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea09763-0611-4be7-b559-0db9dad31c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.203481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1ea09763-0611-4be7-b559-0db9dad31c04 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:07.204993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1ea09763-0611-4be7-b559-0db9dad31c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:07.299178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129b285-9516-4499-a223-a6ecad2500c7 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:07.323546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 135/268] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:07.324426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09763-0611-4be7-b559-0db9dad31c04 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:07.336386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf9b92-d4b0-48b9-b2d6-2e6dba94dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:07.337348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9b92-d4b0-48b9-b2d6-2e6dba94dc69 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.337657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9b92-d4b0-48b9-b2d6-2e6dba94dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.337957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9b92-d4b0-48b9-b2d6-2e6dba94dc69 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.339279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf9b92-d4b0-48b9-b2d6-2e6dba94dc69 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:07.340938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf9b92-d4b0-48b9-b2d6-2e6dba94dc69 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:07.342715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 134/269] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:07 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:07.344083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cadd4-a430-4ea1-b0aa-6444bc359f8e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.344467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cadd4-a430-4ea1-b0aa-6444bc359f8e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.344746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cadd4-a430-4ea1-b0aa-6444bc359f8e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.346026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 135/270] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:07.350816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cadd4-a430-4ea1-b0aa-6444bc359f8e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:07.352020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:07.407122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:07.440321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea09763-0611-4be7-b559-0db9dad31c04 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:07.463185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cadd4-a430-4ea1-b0aa-6444bc359f8e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:07.476559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fd1d2-d683-4a9a-86d6-39ed8ec2a5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:07.477240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd1d2-d683-4a9a-86d6-39ed8ec2a5a2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.477611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd1d2-d683-4a9a-86d6-39ed8ec2a5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.477915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd1d2-d683-4a9a-86d6-39ed8ec2a5a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.478899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fd1d2-d683-4a9a-86d6-39ed8ec2a5a2 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=20a1163a83aa4705a89b4635d4e037b4, user.id=86022e82ef514776b40755664a1f2ccf) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:07.479238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fd1d2-d683-4a9a-86d6-39ed8ec2a5a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:07.480759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fd1d2-d683-4a9a-86d6-39ed8ec2a5a2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:07.493955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 136/271] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:07 2022] GET /identity/v3/role_assignments?user.id=86022e82ef514776b40755664a1f2ccf&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:07.495930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46137eb7-855a-4d47-9daa-bd164d22f737 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.496217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46137eb7-855a-4d47-9daa-bd164d22f737 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.496489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46137eb7-855a-4d47-9daa-bd164d22f737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.500179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:07.500850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.501132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.501414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.502335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] RBAC: Adding query filter params (role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=20a1163a83aa4705a89b4635d4e037b4, user.id=86022e82ef514776b40755664a1f2ccf) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:07.502751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:07.504240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:07.511943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 136/272] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:07 2022] GET /identity/v3/role_assignments?user.id=86022e82ef514776b40755664a1f2ccf&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:07.512907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:07.534931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-46137eb7-855a-4d47-9daa-bd164d22f737 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:07.536289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-46137eb7-855a-4d47-9daa-bd164d22f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:07.642136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c689dc-04b8-421c-8009-2ef28fbfd6bc None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:07.661101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:07.691406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 137/273] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:07.693294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348504-2d2d-45a5-b5a1-227b199b71bd None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.693861 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348504-2d2d-45a5-b5a1-227b199b71bd None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.694339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348504-2d2d-45a5-b5a1-227b199b71bd None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.696351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 138/274] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:07.702702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91421e73-c524-414a-86b0-77bcd9caf6d0 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.703371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91421e73-c524-414a-86b0-77bcd9caf6d0 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.703845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91421e73-c524-414a-86b0-77bcd9caf6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.716436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55833a8f-94ce-47e3-b3b8-835fb28771e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:07.716436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55833a8f-94ce-47e3-b3b8-835fb28771e0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.716436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55833a8f-94ce-47e3-b3b8-835fb28771e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.716436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55833a8f-94ce-47e3-b3b8-835fb28771e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.735014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-91421e73-c524-414a-86b0-77bcd9caf6d0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:07.736235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-91421e73-c524-414a-86b0-77bcd9caf6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:07.736411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55833a8f-94ce-47e3-b3b8-835fb28771e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-55833a8f-94ce-47e3-b3b8-835fb28771e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:07.744979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:07.746124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:07.746902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:07.747512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 137/275] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:07.748301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55833a8f-94ce-47e3-b3b8-835fb28771e0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:07.762158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:07.836816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 139/276] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:07.837846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91421e73-c524-414a-86b0-77bcd9caf6d0 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:07.885892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55833a8f-94ce-47e3-b3b8-835fb28771e0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:07.928918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ac266-89b0-44e7-b45c-23469ac75aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:07.929858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ac266-89b0-44e7-b45c-23469ac75aac None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:07.930095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ac266-89b0-44e7-b45c-23469ac75aac None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:07.930373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ac266-89b0-44e7-b45c-23469ac75aac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:07.941482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ac266-89b0-44e7-b45c-23469ac75aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-865ac266-89b0-44e7-b45c-23469ac75aac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:07.949745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:07.950761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:07.951558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 138/277] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:07.952773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ac266-89b0-44e7-b45c-23469ac75aac None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:07.962978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91421e73-c524-414a-86b0-77bcd9caf6d0 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:07.985576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:08.000899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.001591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.001907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.002190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.002951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:08.003263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.004683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.017656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 140/278] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:08.017656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:08.028905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:08.074439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ac266-89b0-44e7-b45c-23469ac75aac None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:08.112191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523579-c887-4fa2-a6f5-71ffd16e81fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.113013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523579-c887-4fa2-a6f5-71ffd16e81fe None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.113295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523579-c887-4fa2-a6f5-71ffd16e81fe None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.113575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523579-c887-4fa2-a6f5-71ffd16e81fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.114793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48523579-c887-4fa2-a6f5-71ffd16e81fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.116277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48523579-c887-4fa2-a6f5-71ffd16e81fe None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.118105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 139/279] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:08.118713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523579-c887-4fa2-a6f5-71ffd16e81fe None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:08.131080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e5b46-daed-4e53-809c-99bfa4fdf4e2 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:08.171182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.171868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.172175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.172425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.173186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:08.173493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.174929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.184704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 141/280] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:08.185507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:08.225816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:08.234649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523579-c887-4fa2-a6f5-71ffd16e81fe None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:08.273675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:08.279805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e806aae-6d5b-48c5-a76d-f50b929de578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.280699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e806aae-6d5b-48c5-a76d-f50b929de578 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.280979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e806aae-6d5b-48c5-a76d-f50b929de578 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.281637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e806aae-6d5b-48c5-a76d-f50b929de578 None admin] PATH_INFO: `/v3/users/68684a19709847ae8e9a20a187039cf1` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.297238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e806aae-6d5b-48c5-a76d-f50b929de578 None admin] RBAC: Authorizing `identity:get_user(user_id=68684a19709847ae8e9a20a187039cf1)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.298693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e806aae-6d5b-48c5-a76d-f50b929de578 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.300821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 140/281] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:08 2022] GET /identity/v3/users/68684a19709847ae8e9a20a187039cf1 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:08.301590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e806aae-6d5b-48c5-a76d-f50b929de578 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:08.310359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf45330-c843-433c-b904-8eb31c76b1ef None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:08.346538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa663af-dcde-4127-a726-58236d0d6122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.347194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa663af-dcde-4127-a726-58236d0d6122 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.347480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa663af-dcde-4127-a726-58236d0d6122 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.347756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa663af-dcde-4127-a726-58236d0d6122 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.348470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa663af-dcde-4127-a726-58236d0d6122 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.349966 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa663af-dcde-4127-a726-58236d0d6122 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.373494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 142/282] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:40:08 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:08.374490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa663af-dcde-4127-a726-58236d0d6122 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:08.445695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e806aae-6d5b-48c5-a76d-f50b929de578 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:08.500448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d37c93-f501-4d58-9b73-64d915832129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.502476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d37c93-f501-4d58-9b73-64d915832129 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.503243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d37c93-f501-4d58-9b73-64d915832129 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.503762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d37c93-f501-4d58-9b73-64d915832129 None admin] PATH_INFO: `/v3/users/823c2d577e864ab38f954db574c67187` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.506257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d37c93-f501-4d58-9b73-64d915832129 None admin] RBAC: Authorizing `identity:get_user(user_id=823c2d577e864ab38f954db574c67187)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.508009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa663af-dcde-4127-a726-58236d0d6122 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:08.508244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d37c93-f501-4d58-9b73-64d915832129 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.511052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 141/283] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:08 2022] GET /identity/v3/users/823c2d577e864ab38f954db574c67187 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:08.518957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d37c93-f501-4d58-9b73-64d915832129 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:08.546240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:08.550424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:08.564015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.564694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.564975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.565256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.566911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] RBAC: Adding query filter params (role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, user.id=68684a19709847ae8e9a20a187039cf1, scope.system=all) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:08.566911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.567954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.574498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 143/284] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Jun 21 15:40:08 2022] GET /identity/v3/role_assignments?user.id=68684a19709847ae8e9a20a187039cf1&scope.system=all&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:08.659018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d37c93-f501-4d58-9b73-64d915832129 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:08.701275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26910b0-a55e-4034-897f-6b32556ed681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.702346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26910b0-a55e-4034-897f-6b32556ed681 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.702632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26910b0-a55e-4034-897f-6b32556ed681 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.702914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26910b0-a55e-4034-897f-6b32556ed681 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.704236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26910b0-a55e-4034-897f-6b32556ed681 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.705726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26910b0-a55e-4034-897f-6b32556ed681 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.707537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 142/285] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:08 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:08.713596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:08.792649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825057ac-cf63-43a9-8bff-0be8efcd4e99 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.792942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825057ac-cf63-43a9-8bff-0be8efcd4e99 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.793126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825057ac-cf63-43a9-8bff-0be8efcd4e99 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.794903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 143/286] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:40:08.800100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b748a-1815-459a-9d6d-075d8eeb0fed None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.800390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b748a-1815-459a-9d6d-075d8eeb0fed None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.800674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b748a-1815-459a-9d6d-075d8eeb0fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.827053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e08b748a-1815-459a-9d6d-075d8eeb0fed None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:08.828347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e08b748a-1815-459a-9d6d-075d8eeb0fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:08.836637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0752e-5e9e-4eea-ae1f-4e89d102d646 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:08.879743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044de61-378f-46c0-988c-124b86409e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:08.880673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044de61-378f-46c0-988c-124b86409e3a None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:08.880958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044de61-378f-46c0-988c-124b86409e3a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:08.881222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044de61-378f-46c0-988c-124b86409e3a None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4/users/823c2d577e864ab38f954db574c67187/roles/24a12434807d480fafb72632fc2d6756` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:08.890792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044de61-378f-46c0-988c-124b86409e3a None admin] RBAC: Authorizing `identity:create_grant(project_id=20a1163a83aa4705a89b4635d4e037b4, user_id=823c2d577e864ab38f954db574c67187, role_id=24a12434807d480fafb72632fc2d6756)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:08.892286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044de61-378f-46c0-988c-124b86409e3a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:08.924915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 144/287] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:08 2022] PUT /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4/users/823c2d577e864ab38f954db574c67187/roles/24a12434807d480fafb72632fc2d6756 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:08.940903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 144/288] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:08.982508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044de61-378f-46c0-988c-124b86409e3a None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:09.066869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:09.138053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044de61-378f-46c0-988c-124b86409e3a None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:09.178146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792c7c6-dfe6-40f5-bc19-f2d20b49001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:09.178973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c7c6-dfe6-40f5-bc19-f2d20b49001f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:09.179259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c7c6-dfe6-40f5-bc19-f2d20b49001f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:09.179537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c7c6-dfe6-40f5-bc19-f2d20b49001f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:09.192433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792c7c6-dfe6-40f5-bc19-f2d20b49001f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-a792c7c6-dfe6-40f5-bc19-f2d20b49001f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:09.200736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:09.201759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:09.202588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 145/289] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:09.209651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b748a-1815-459a-9d6d-075d8eeb0fed None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:09.237262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:09.329097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b748a-1815-459a-9d6d-075d8eeb0fed None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:09.365080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:09.365760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:09.366065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:09.366343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:09.367091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] RBAC: Adding query filter params (name=member) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:09.367396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:09.368781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:09.378256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 145/290] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:09.383456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792c7c6-dfe6-40f5-bc19-f2d20b49001f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:09.537556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792c7c6-dfe6-40f5-bc19-f2d20b49001f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:09.563240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:09.586791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b2a20-ad5b-4f60-8553-0cc5ade1efe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:09.587990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b2a20-ad5b-4f60-8553-0cc5ade1efe1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:09.588520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b2a20-ad5b-4f60-8553-0cc5ade1efe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:09.589074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b2a20-ad5b-4f60-8553-0cc5ade1efe1 None admin] PATH_INFO: `/v3/users/0dd7ab851f154030963985c6a4f7d3e4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:09.590599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b2a20-ad5b-4f60-8553-0cc5ade1efe1 None admin] RBAC: Authorizing `identity:get_user(user_id=0dd7ab851f154030963985c6a4f7d3e4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:09.597797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b2a20-ad5b-4f60-8553-0cc5ade1efe1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:09.600164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 146/291] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:09 2022] GET /identity/v3/users/0dd7ab851f154030963985c6a4f7d3e4 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:09.606092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:09.707769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:09.728666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56434387-4ea9-4e68-b47e-989752e3cdec None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:09.765109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5f767-1d57-4746-9970-d5cb0b3065cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:09.765766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5f767-1d57-4746-9970-d5cb0b3065cf None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:09.766077 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5f767-1d57-4746-9970-d5cb0b3065cf None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:09.766349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5f767-1d57-4746-9970-d5cb0b3065cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:09.837543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5f767-1d57-4746-9970-d5cb0b3065cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:09.839530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5f767-1d57-4746-9970-d5cb0b3065cf None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:09.926020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:09.947970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 146/292] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:09.953804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b2a20-ad5b-4f60-8553-0cc5ade1efe1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:10.122979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b2a20-ad5b-4f60-8553-0cc5ade1efe1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:10.180292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2d75b-201d-4a85-816b-6f9f6cd7e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:10.181176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2d75b-201d-4a85-816b-6f9f6cd7e36a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:10.181457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2d75b-201d-4a85-816b-6f9f6cd7e36a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:10.181729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2d75b-201d-4a85-816b-6f9f6cd7e36a None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:10.183220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2d75b-201d-4a85-816b-6f9f6cd7e36a None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:10.184691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2d75b-201d-4a85-816b-6f9f6cd7e36a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:10.186814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 147/293] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:09 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:10.192525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5f767-1d57-4746-9970-d5cb0b3065cf None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:10.302712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5f767-1d57-4746-9970-d5cb0b3065cf None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:10.338589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87f78b-b6b8-4354-bac4-532fcc65cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:10.339835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87f78b-b6b8-4354-bac4-532fcc65cd10 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:10.340119 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87f78b-b6b8-4354-bac4-532fcc65cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:10.340396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87f78b-b6b8-4354-bac4-532fcc65cd10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:10.341238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87f78b-b6b8-4354-bac4-532fcc65cd10 None admin] RBAC: Adding query filter params (role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=20a1163a83aa4705a89b4635d4e037b4, user.id=0dd7ab851f154030963985c6a4f7d3e4) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:10.341547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87f78b-b6b8-4354-bac4-532fcc65cd10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:10.343019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87f78b-b6b8-4354-bac4-532fcc65cd10 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:10.349309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 147/294] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:10 2022] GET /identity/v3/role_assignments?user.id=0dd7ab851f154030963985c6a4f7d3e4&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:11.115152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3700e-8bd7-4bab-b640-99dd9d048f08 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.115425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3700e-8bd7-4bab-b640-99dd9d048f08 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.115693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3700e-8bd7-4bab-b640-99dd9d048f08 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.117315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 148/295] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:11.119898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cabb5-467b-4317-b335-a63f255d0079 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.121134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cabb5-467b-4317-b335-a63f255d0079 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.121683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cabb5-467b-4317-b335-a63f255d0079 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.123142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ca7ad-57dd-4109-82e0-f216640f2ce0 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.123323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 148/296] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:11.123496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ca7ad-57dd-4109-82e0-f216640f2ce0 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.123838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ca7ad-57dd-4109-82e0-f216640f2ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.135618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba853-5d9e-480e-b410-31658cc5bb90 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.136094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba853-5d9e-480e-b410-31658cc5bb90 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.136094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eba853-5d9e-480e-b410-31658cc5bb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.150313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5a0ca7ad-57dd-4109-82e0-f216640f2ce0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:11.152723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5a0ca7ad-57dd-4109-82e0-f216640f2ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:11.153204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d2eba853-5d9e-480e-b410-31658cc5bb90 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:11.154383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d2eba853-5d9e-480e-b410-31658cc5bb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:11.202101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:11.206829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:11.236688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 149/297] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:11.242541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 149/298] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:11.254896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65748d0e-6c9b-40cd-a0c3-c51a84288714 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.255402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65748d0e-6c9b-40cd-a0c3-c51a84288714 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.255869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65748d0e-6c9b-40cd-a0c3-c51a84288714 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.257701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 150/299] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:11.262960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc146d4-5a3a-4045-ba5f-9fc08318bf7c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.263252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc146d4-5a3a-4045-ba5f-9fc08318bf7c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.263699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc146d4-5a3a-4045-ba5f-9fc08318bf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.266533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65748d0e-6c9b-40cd-a0c3-c51a84288714 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:11.281697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2fc146d4-5a3a-4045-ba5f-9fc08318bf7c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:11.282961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2fc146d4-5a3a-4045-ba5f-9fc08318bf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:11.360191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 150/300] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:11.362990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc146d4-5a3a-4045-ba5f-9fc08318bf7c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:11.380266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65748d0e-6c9b-40cd-a0c3-c51a84288714 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:11.417104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bd3aa-8f28-4111-a4a4-97c102c6ed7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:11.417973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bd3aa-8f28-4111-a4a4-97c102c6ed7e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.418257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bd3aa-8f28-4111-a4a4-97c102c6ed7e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.418530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bd3aa-8f28-4111-a4a4-97c102c6ed7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.428651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bd3aa-8f28-4111-a4a4-97c102c6ed7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-4b0bd3aa-8f28-4111-a4a4-97c102c6ed7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:11.436626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:11.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:11.438652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 151/301] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:11.439295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6add7a-ff72-4cf9-90aa-c6ec406aff2d None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.439295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6add7a-ff72-4cf9-90aa-c6ec406aff2d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.439595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6add7a-ff72-4cf9-90aa-c6ec406aff2d None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.441073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 152/302] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:11.441629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6add7a-ff72-4cf9-90aa-c6ec406aff2d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:11.475686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc146d4-5a3a-4045-ba5f-9fc08318bf7c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:11.478547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:11.485800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:11.511613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79627fa5-bb1c-4038-b03f-546f934a92ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:11.512419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79627fa5-bb1c-4038-b03f-546f934a92ca None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.512700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79627fa5-bb1c-4038-b03f-546f934a92ca None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.512983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79627fa5-bb1c-4038-b03f-546f934a92ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.524953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79627fa5-bb1c-4038-b03f-546f934a92ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-79627fa5-bb1c-4038-b03f-546f934a92ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:11.533719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:11.534617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:11.535370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 151/303] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:11.536002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79627fa5-bb1c-4038-b03f-546f934a92ca None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:11.549681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6add7a-ff72-4cf9-90aa-c6ec406aff2d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:11.584774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efbad8-2011-4b99-ba59-d9d058730b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:11.585575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efbad8-2011-4b99-ba59-d9d058730b09 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.585888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efbad8-2011-4b99-ba59-d9d058730b09 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.586170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efbad8-2011-4b99-ba59-d9d058730b09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.587357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efbad8-2011-4b99-ba59-d9d058730b09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:11.588938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efbad8-2011-4b99-ba59-d9d058730b09 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:11.590596 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 153/304] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:11.591741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58714-3bd1-4add-8620-a83685fef434 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.592094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58714-3bd1-4add-8620-a83685fef434 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.592298 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd58714-3bd1-4add-8620-a83685fef434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.608361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3fd58714-3bd1-4add-8620-a83685fef434 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:11.609567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3fd58714-3bd1-4add-8620-a83685fef434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:11.642990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79627fa5-bb1c-4038-b03f-546f934a92ca None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:11.676709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:11.677384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.677670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.677985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.678754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:11.679061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:11.686029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 154/305] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:11.686178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:11.686290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58714-3bd1-4add-8620-a83685fef434 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:11.689115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 152/306] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:11.689944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:11.719538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:11.727265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:11.789185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd58714-3bd1-4add-8620-a83685fef434 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:11.800944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d7829-d523-4d05-829d-246a83afccba None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:11.828599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:11.829277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.829626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.829948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.830711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:11.831022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:11.832422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:11.840430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58506-0a3e-46a7-b670-44eb21a00140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:11.841111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58506-0a3e-46a7-b670-44eb21a00140 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:11.841395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58506-0a3e-46a7-b670-44eb21a00140 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:11.841606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 155/307] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:11.841697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58506-0a3e-46a7-b670-44eb21a00140 None admin] PATH_INFO: `/v3/groups` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:11.842440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:11.846871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d58506-0a3e-46a7-b670-44eb21a00140 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:11.848336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d58506-0a3e-46a7-b670-44eb21a00140 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:11.864584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 153/308] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Jun 21 15:40:11 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:11.865619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58506-0a3e-46a7-b670-44eb21a00140 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:11.954571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:11.963029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdde4f-7137-48a9-bafc-afabafd11ae6 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:11.984934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:11.998916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58506-0a3e-46a7-b670-44eb21a00140 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:12.003223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7781694-5ca5-45a2-9fe2-c0e000d6346f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:12.004043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7781694-5ca5-45a2-9fe2-c0e000d6346f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:12.004324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7781694-5ca5-45a2-9fe2-c0e000d6346f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:12.004600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7781694-5ca5-45a2-9fe2-c0e000d6346f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:12.018587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7781694-5ca5-45a2-9fe2-c0e000d6346f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-a7781694-5ca5-45a2-9fe2-c0e000d6346f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:12.028059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:12.029190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:12.030000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 156/309] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:12.030644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7781694-5ca5-45a2-9fe2-c0e000d6346f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:12.043785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00961fe3-f345-4857-a7dc-c04169218840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:12.044357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00961fe3-f345-4857-a7dc-c04169218840 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:12.045129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00961fe3-f345-4857-a7dc-c04169218840 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:12.045415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00961fe3-f345-4857-a7dc-c04169218840 None admin] PATH_INFO: `/v3/users/68684a19709847ae8e9a20a187039cf1` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:12.046712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00961fe3-f345-4857-a7dc-c04169218840 None admin] RBAC: Authorizing `identity:get_user(user_id=68684a19709847ae8e9a20a187039cf1)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:12.048193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00961fe3-f345-4857-a7dc-c04169218840 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:12.050158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 154/310] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:11 2022] GET /identity/v3/users/68684a19709847ae8e9a20a187039cf1 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:12.050996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00961fe3-f345-4857-a7dc-c04169218840 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:12.152959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7781694-5ca5-45a2-9fe2-c0e000d6346f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:12.164558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00961fe3-f345-4857-a7dc-c04169218840 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:12.188248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac66535-2567-4f14-9622-067f66678db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:12.189172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac66535-2567-4f14-9622-067f66678db5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:12.189457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac66535-2567-4f14-9622-067f66678db5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:12.189723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac66535-2567-4f14-9622-067f66678db5 None admin] PATH_INFO: `/v3/users/03bb8b8238a34e00ab0c6157785b914a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:12.200801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:12.201487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:12.201785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:12.202400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:12.202954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:12.203264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:12.204743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:12.205087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac66535-2567-4f14-9622-067f66678db5 None admin] RBAC: Authorizing `identity:get_user(user_id=03bb8b8238a34e00ab0c6157785b914a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:12.206595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac66535-2567-4f14-9622-067f66678db5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:12.208514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 157/311] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:12 2022] GET /identity/v3/users/03bb8b8238a34e00ab0c6157785b914a => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:12.209094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac66535-2567-4f14-9622-067f66678db5 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:12.218738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 155/312] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:12.219553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:12.231122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:12.246184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:12.329155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4faf09-d986-41f6-8c19-1c67fda116ad None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:12.332503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac66535-2567-4f14-9622-067f66678db5 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:12.367021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:12.367692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:12.367971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:12.368242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:12.369101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] RBAC: Adding query filter params (role.id=24a12434807d480fafb72632fc2d6756, user.id=03bb8b8238a34e00ab0c6157785b914a, scope.system=all) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:12.369412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:12.371341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:12.381794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 156/313] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Jun 21 15:40:12 2022] GET /identity/v3/role_assignments?user.id=03bb8b8238a34e00ab0c6157785b914a&scope.system=all&role.id=24a12434807d480fafb72632fc2d6756 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:12.381794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:12.383390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabf0d0-025f-4990-b866-2127cd815723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:12.384400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabf0d0-025f-4990-b866-2127cd815723 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:12.384685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabf0d0-025f-4990-b866-2127cd815723 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:12.384961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabf0d0-025f-4990-b866-2127cd815723 None admin] PATH_INFO: `/v3/system/users/68684a19709847ae8e9a20a187039cf1/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:12.386642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabf0d0-025f-4990-b866-2127cd815723 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=68684a19709847ae8e9a20a187039cf1, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:12.388069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabf0d0-025f-4990-b866-2127cd815723 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:12.400851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 158/314] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Jun 21 15:40:12 2022] PUT /identity/v3/system/users/68684a19709847ae8e9a20a187039cf1/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:12.501005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:12.519239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234431a4-30ea-44ad-bb87-a1322d2a9f65 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:12.560906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193239be-b1f8-4e76-8226-65a0eac5e1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:12.561867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193239be-b1f8-4e76-8226-65a0eac5e1de None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:12.562151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193239be-b1f8-4e76-8226-65a0eac5e1de None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:12.562426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193239be-b1f8-4e76-8226-65a0eac5e1de None admin] PATH_INFO: `/v3/users/823c2d577e864ab38f954db574c67187` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:12.563763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193239be-b1f8-4e76-8226-65a0eac5e1de None admin] RBAC: Authorizing `identity:get_user(user_id=823c2d577e864ab38f954db574c67187)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:12.565299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193239be-b1f8-4e76-8226-65a0eac5e1de None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:12.567611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 157/315] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:12 2022] GET /identity/v3/users/823c2d577e864ab38f954db574c67187 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:12.573997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabf0d0-025f-4990-b866-2127cd815723 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:12.669970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:12.694703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabf0d0-025f-4990-b866-2127cd815723 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:12.730180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709565-204c-4488-ae91-4f60488ee47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:12.731339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709565-204c-4488-ae91-4f60488ee47d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:12.731628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709565-204c-4488-ae91-4f60488ee47d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:12.732393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709565-204c-4488-ae91-4f60488ee47d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:12.804654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a709565-204c-4488-ae91-4f60488ee47d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:12.806095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a709565-204c-4488-ae91-4f60488ee47d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:12.890336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:12.911944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 159/316] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:12.918192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193239be-b1f8-4e76-8226-65a0eac5e1de None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:13.032401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193239be-b1f8-4e76-8226-65a0eac5e1de None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:13.070539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fba9df-373d-439a-b91a-fd8aa5a4bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:13.071336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba9df-373d-439a-b91a-fd8aa5a4bd78 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:13.071621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba9df-373d-439a-b91a-fd8aa5a4bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:13.071903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fba9df-373d-439a-b91a-fd8aa5a4bd78 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:13.073358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fba9df-373d-439a-b91a-fd8aa5a4bd78 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:13.074772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fba9df-373d-439a-b91a-fd8aa5a4bd78 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:13.076701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 158/317] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:12 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:13.082645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709565-204c-4488-ae91-4f60488ee47d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:13.188840 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709565-204c-4488-ae91-4f60488ee47d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:13.224781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97bf9b-b57b-4e94-bf71-510c308752f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:13.225457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97bf9b-b57b-4e94-bf71-510c308752f1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:13.225741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97bf9b-b57b-4e94-bf71-510c308752f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:13.226050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97bf9b-b57b-4e94-bf71-510c308752f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:13.226901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97bf9b-b57b-4e94-bf71-510c308752f1 None admin] RBAC: Adding query filter params (role.id=24a12434807d480fafb72632fc2d6756, scope.project.id=20a1163a83aa4705a89b4635d4e037b4, user.id=823c2d577e864ab38f954db574c67187) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:13.227215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97bf9b-b57b-4e94-bf71-510c308752f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:13.228662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97bf9b-b57b-4e94-bf71-510c308752f1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:13.241956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 160/318] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:13 2022] GET /identity/v3/role_assignments?user.id=823c2d577e864ab38f954db574c67187&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=24a12434807d480fafb72632fc2d6756 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:13.971364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2ff3f-58d8-4b86-87f5-5c376b7ee65f None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:13.971909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2ff3f-58d8-4b86-87f5-5c376b7ee65f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:13.971909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2ff3f-58d8-4b86-87f5-5c376b7ee65f None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:13.973243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 159/319] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:13.978694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db32dfd-fdd4-4cec-8d12-9e5228858f26 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:13.979139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db32dfd-fdd4-4cec-8d12-9e5228858f26 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:13.979426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db32dfd-fdd4-4cec-8d12-9e5228858f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:13.996899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-9db32dfd-fdd4-4cec-8d12-9e5228858f26 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:13.998082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9db32dfd-fdd4-4cec-8d12-9e5228858f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:14.035924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:14.071046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 161/320] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:14.093711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2ff3f-58d8-4b86-87f5-5c376b7ee65f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:14.117187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:14.200346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2ff3f-58d8-4b86-87f5-5c376b7ee65f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:14.253545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8643dd-411a-4685-8987-a4cadf59df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:14.254548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8643dd-411a-4685-8987-a4cadf59df76 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:14.254828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8643dd-411a-4685-8987-a4cadf59df76 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:14.255101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8643dd-411a-4685-8987-a4cadf59df76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:14.256390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8643dd-411a-4685-8987-a4cadf59df76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:14.257918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8643dd-411a-4685-8987-a4cadf59df76 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:14.259738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 160/321] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:14.265587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db32dfd-fdd4-4cec-8d12-9e5228858f26 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:14.375316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db32dfd-fdd4-4cec-8d12-9e5228858f26 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:14.411857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2eddc2-591b-4e38-9a65-65642e8d41c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:14.412540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eddc2-591b-4e38-9a65-65642e8d41c1 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:14.412824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eddc2-591b-4e38-9a65-65642e8d41c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:14.413100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eddc2-591b-4e38-9a65-65642e8d41c1 None admin] PATH_INFO: `/v3/groups` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:14.413812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2eddc2-591b-4e38-9a65-65642e8d41c1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:14.415330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2eddc2-591b-4e38-9a65-65642e8d41c1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:14.434375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 162/322] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Jun 21 15:40:14 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:14.535489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf3d0a-7c44-4c48-8ce2-545fb0e7c179 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:14.535663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf3d0a-7c44-4c48-8ce2-545fb0e7c179 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:14.535900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf3d0a-7c44-4c48-8ce2-545fb0e7c179 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:14.537415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 161/323] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:14.543142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a48842-6137-4426-812a-b3758c4167c4 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:14.543427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a48842-6137-4426-812a-b3758c4167c4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:14.543704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a48842-6137-4426-812a-b3758c4167c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:14.550891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d914de4-aa9f-4c6c-9957-aa06e0c545c0 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:14.551272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d914de4-aa9f-4c6c-9957-aa06e0c545c0 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:14.551612 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d914de4-aa9f-4c6c-9957-aa06e0c545c0 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:14.553111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 162/324] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:40:14.559212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ddb90-d8c6-4694-8e48-0b98dbb5f36d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:14.559496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ddb90-d8c6-4694-8e48-0b98dbb5f36d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:14.559746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ddb90-d8c6-4694-8e48-0b98dbb5f36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:14.566199 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-55a48842-6137-4426-812a-b3758c4167c4 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:14.567407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-55a48842-6137-4426-812a-b3758c4167c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:14.577112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-418ddb90-d8c6-4694-8e48-0b98dbb5f36d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:14.578377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-418ddb90-d8c6-4694-8e48-0b98dbb5f36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:14.596774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:14.659090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 163/325] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:14.661200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 163/326] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:14.682774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a48842-6137-4426-812a-b3758c4167c4 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:14.684340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ddb90-d8c6-4694-8e48-0b98dbb5f36d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:14.703523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:14.816153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ddb90-d8c6-4694-8e48-0b98dbb5f36d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:14.825512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a48842-6137-4426-812a-b3758c4167c4 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:14.868536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1bda0-143a-4017-87fb-04af9f12a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:14.869431 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1bda0-143a-4017-87fb-04af9f12a871 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:14.869752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1bda0-143a-4017-87fb-04af9f12a871 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:14.870427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1bda0-143a-4017-87fb-04af9f12a871 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:14.886868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1bda0-143a-4017-87fb-04af9f12a871 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:14.890425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c8688-57ae-4154-9147-fe7ab530c344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:14.891517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c8688-57ae-4154-9147-fe7ab530c344 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:14.891986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c8688-57ae-4154-9147-fe7ab530c344 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:14.892445 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c8688-57ae-4154-9147-fe7ab530c344 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-56a1bda0-143a-4017-87fb-04af9f12a871 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:14.895715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:14.896604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:14.897384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 164/327] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:14.903908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1bda0-143a-4017-87fb-04af9f12a871 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:14.911840 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c8688-57ae-4154-9147-fe7ab530c344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-b40c8688-57ae-4154-9147-fe7ab530c344 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:14.922799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:14.923684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:14.924465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:14.928032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 164/328] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:14.934312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c8688-57ae-4154-9147-fe7ab530c344 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:14.965210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:15.021147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:15.056041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1bda0-143a-4017-87fb-04af9f12a871 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:15.069668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c8688-57ae-4154-9147-fe7ab530c344 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:15.096442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:15.097370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:15.097826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:15.098270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:15.099162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:15.099602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:15.101208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:15.107506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:15.108556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:15.108556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:15.108803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:15.109556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] RBAC: Adding query filter params (name=member) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:15.109892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:15.111325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:15.112907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 165/329] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:15.122189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 165/330] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:15.123308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:15.128990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:15.256929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509f1fa-2fd9-4779-9f29-b57d49f193a7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:15.277162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:15.307670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945bb7a-4339-4aef-9cdc-60aac9dbf678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:15.308786 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945bb7a-4339-4aef-9cdc-60aac9dbf678 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:15.309067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945bb7a-4339-4aef-9cdc-60aac9dbf678 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:15.309346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945bb7a-4339-4aef-9cdc-60aac9dbf678 None admin] PATH_INFO: `/v3/users/03bb8b8238a34e00ab0c6157785b914a` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:15.312126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c67672c-b45c-47e7-98a1-f463275dd53c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:15.314821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945bb7a-4339-4aef-9cdc-60aac9dbf678 None admin] RBAC: Authorizing `identity:get_user(user_id=03bb8b8238a34e00ab0c6157785b914a)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:15.316533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945bb7a-4339-4aef-9cdc-60aac9dbf678 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:15.318856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 166/331] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:15 2022] GET /identity/v3/users/03bb8b8238a34e00ab0c6157785b914a => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:15.335640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945bb7a-4339-4aef-9cdc-60aac9dbf678 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:15.351776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:15.359607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce87e1-7e65-4954-a250-de0ad424c9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:15.361157 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce87e1-7e65-4954-a250-de0ad424c9b2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:15.361441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce87e1-7e65-4954-a250-de0ad424c9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:15.361720 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce87e1-7e65-4954-a250-de0ad424c9b2 None admin] PATH_INFO: `/v3/users/68684a19709847ae8e9a20a187039cf1` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:15.363310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce87e1-7e65-4954-a250-de0ad424c9b2 None admin] RBAC: Authorizing `identity:get_user(user_id=68684a19709847ae8e9a20a187039cf1)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:15.365970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce87e1-7e65-4954-a250-de0ad424c9b2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:15.367553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 166/332] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:15 2022] GET /identity/v3/users/68684a19709847ae8e9a20a187039cf1 => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:15.373008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce87e1-7e65-4954-a250-de0ad424c9b2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:15.478189 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945bb7a-4339-4aef-9cdc-60aac9dbf678 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:15.514387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce87e1-7e65-4954-a250-de0ad424c9b2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:15.532199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b27f16-2d66-49a0-a7c7-e6204016f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:15.533103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27f16-2d66-49a0-a7c7-e6204016f8e0 None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:15.533389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27f16-2d66-49a0-a7c7-e6204016f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:15.533670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27f16-2d66-49a0-a7c7-e6204016f8e0 None admin] PATH_INFO: `/v3/system/users/03bb8b8238a34e00ab0c6157785b914a/roles/24a12434807d480fafb72632fc2d6756` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:15.535579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b27f16-2d66-49a0-a7c7-e6204016f8e0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03bb8b8238a34e00ab0c6157785b914a, role_id=24a12434807d480fafb72632fc2d6756)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:15.537634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b27f16-2d66-49a0-a7c7-e6204016f8e0 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:15.557801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 167/333] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Jun 21 15:40:15 2022] PUT /identity/v3/system/users/03bb8b8238a34e00ab0c6157785b914a/roles/24a12434807d480fafb72632fc2d6756 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:40:15.586808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebbe6d-513b-435b-9a64-8b3c70e1a0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:15.587777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebbe6d-513b-435b-9a64-8b3c70e1a0d7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:15.588289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebbe6d-513b-435b-9a64-8b3c70e1a0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:15.588797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebbe6d-513b-435b-9a64-8b3c70e1a0d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:15.590000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebbe6d-513b-435b-9a64-8b3c70e1a0d7 None admin] RBAC: Adding query filter params (role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, user.id=68684a19709847ae8e9a20a187039cf1, scope.system=all) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:15.590471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebbe6d-513b-435b-9a64-8b3c70e1a0d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:15.592508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebbe6d-513b-435b-9a64-8b3c70e1a0d7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:15.610056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 167/334] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Jun 21 15:40:15 2022] GET /identity/v3/role_assignments?user.id=68684a19709847ae8e9a20a187039cf1&scope.system=all&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 510 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:16.995571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc9e7-d7de-4017-a006-5d50d954adf8 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:16.996244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc9e7-d7de-4017-a006-5d50d954adf8 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:16.996392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc9e7-d7de-4017-a006-5d50d954adf8 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:16.998049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 168/335] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:17.016007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e01f-0a5b-49fd-927d-2ce0397cbffa None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:17.016518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e01f-0a5b-49fd-927d-2ce0397cbffa None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:17.016591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e01f-0a5b-49fd-927d-2ce0397cbffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:17.042923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0241e01f-0a5b-49fd-927d-2ce0397cbffa None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:17.044158 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0241e01f-0a5b-49fd-927d-2ce0397cbffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:17.046654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:17.127073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 168/336] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:17.160550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dc9e7-d7de-4017-a006-5d50d954adf8 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:17.198698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba489e10-a257-445f-9802-8c75bc4c5d70 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:17.199002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba489e10-a257-445f-9802-8c75bc4c5d70 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:17.199569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba489e10-a257-445f-9802-8c75bc4c5d70 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:17.200673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 169/337] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:40:17.206073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e18b80-2fe1-4e2e-8083-1758ee9b6e0d None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:17.206417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e18b80-2fe1-4e2e-8083-1758ee9b6e0d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:17.206705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e18b80-2fe1-4e2e-8083-1758ee9b6e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:17.242837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d5e18b80-2fe1-4e2e-8083-1758ee9b6e0d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:17.244034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d5e18b80-2fe1-4e2e-8083-1758ee9b6e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:17.272341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:17.315818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dc9e7-d7de-4017-a006-5d50d954adf8 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:17.386537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25806bef-befa-41dd-8020-1a9b1f117926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:17.387266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 170/338] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:17.388071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25806bef-befa-41dd-8020-1a9b1f117926 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:17.388606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25806bef-befa-41dd-8020-1a9b1f117926 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:17.389094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25806bef-befa-41dd-8020-1a9b1f117926 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:17.414099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e18b80-2fe1-4e2e-8083-1758ee9b6e0d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:17.414577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25806bef-befa-41dd-8020-1a9b1f117926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-25806bef-befa-41dd-8020-1a9b1f117926 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:17.427395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:17.428363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:17.429152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:17.434626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 169/339] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:17.442023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25806bef-befa-41dd-8020-1a9b1f117926 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:17.570580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:17.581305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25806bef-befa-41dd-8020-1a9b1f117926 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:17.614332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:17.618616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e18b80-2fe1-4e2e-8083-1758ee9b6e0d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:17.625307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:17.626501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:17.626986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:17.627458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:17.628462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:17.629032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:17.630667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:17.640731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 170/340] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:17.646527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:17.658539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed0659-fa05-45f6-b2c3-b2d4e1fbe25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:17.659374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed0659-fa05-45f6-b2c3-b2d4e1fbe25f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:17.659658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed0659-fa05-45f6-b2c3-b2d4e1fbe25f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:17.659937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed0659-fa05-45f6-b2c3-b2d4e1fbe25f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:17.671668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed0659-fa05-45f6-b2c3-b2d4e1fbe25f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f5ed0659-fa05-45f6-b2c3-b2d4e1fbe25f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:17.679127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:17.679985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:17.680781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 171/341] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:17.686479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed0659-fa05-45f6-b2c3-b2d4e1fbe25f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:17.764414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da26508-b616-4bc9-b40f-d705aa722840 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:17.810749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e304398-c618-498b-ad68-318b769d65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:17.811877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e304398-c618-498b-ad68-318b769d65cf None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:17.812358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e304398-c618-498b-ad68-318b769d65cf None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:17.812854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e304398-c618-498b-ad68-318b769d65cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:17.872286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed0659-fa05-45f6-b2c3-b2d4e1fbe25f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:17.921205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:17.929978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e304398-c618-498b-ad68-318b769d65cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:17.932035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e304398-c618-498b-ad68-318b769d65cf None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:17.958912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:17.990522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:17.991229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:17.991570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:17.992093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:17.992632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:17.992934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:17.994495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:18.013207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 172/342] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:18.019010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:18.061249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 171/343] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:18.068583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e304398-c618-498b-ad68-318b769d65cf None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:18.233073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9300c-d793-48e0-a02c-044b0a61e1bb None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:18.286738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078ea2-e4e5-46c3-9f5c-47aaa56f10e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:18.288667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ea2-e4e5-46c3-9f5c-47aaa56f10e9 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:18.289147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ea2-e4e5-46c3-9f5c-47aaa56f10e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:18.292086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ea2-e4e5-46c3-9f5c-47aaa56f10e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:18.341100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:18.351569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e304398-c618-498b-ad68-318b769d65cf None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:18.398895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51438ff-9074-47b7-ad18-93b2f4b50e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:18.403991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51438ff-9074-47b7-ad18-93b2f4b50e85 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:18.404544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51438ff-9074-47b7-ad18-93b2f4b50e85 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:18.405032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51438ff-9074-47b7-ad18-93b2f4b50e85 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:18.407250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51438ff-9074-47b7-ad18-93b2f4b50e85 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:18.410750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51438ff-9074-47b7-ad18-93b2f4b50e85 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:18.413536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 172/344] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:18 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:18.417444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8995c-e1b1-46c6-bdeb-de40c4327307 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:18.421267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8995c-e1b1-46c6-bdeb-de40c4327307 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:18.421836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8995c-e1b1-46c6-bdeb-de40c4327307 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:18.426264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 173/345] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:18.427232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8995c-e1b1-46c6-bdeb-de40c4327307 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:18.449740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:18.466587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c078ea2-e4e5-46c3-9f5c-47aaa56f10e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:18.468723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c078ea2-e4e5-46c3-9f5c-47aaa56f10e9 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:18.608754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 173/346] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 15:40:18.610427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e15819-1159-4b1a-b103-ee909b2d49b7 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:18.610973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e15819-1159-4b1a-b103-ee909b2d49b7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:18.611444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e15819-1159-4b1a-b103-ee909b2d49b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:18.640929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f5e15819-1159-4b1a-b103-ee909b2d49b7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:18.642518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f5e15819-1159-4b1a-b103-ee909b2d49b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:18.646562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8995c-e1b1-46c6-bdeb-de40c4327307 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:18.688058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cdea7-6af5-4a06-bc1a-02d2a0bc8edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:18.689364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cdea7-6af5-4a06-bc1a-02d2a0bc8edd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:18.689740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cdea7-6af5-4a06-bc1a-02d2a0bc8edd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:18.690128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cdea7-6af5-4a06-bc1a-02d2a0bc8edd None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:18.697698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cdea7-6af5-4a06-bc1a-02d2a0bc8edd None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:18.699245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cdea7-6af5-4a06-bc1a-02d2a0bc8edd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:18.701962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 174/347] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:18 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:18.703068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cdea7-6af5-4a06-bc1a-02d2a0bc8edd None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:18.739630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:18.795182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 174/348] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:18.796315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e15819-1159-4b1a-b103-ee909b2d49b7 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:18.836706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cdea7-6af5-4a06-bc1a-02d2a0bc8edd None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:18.846323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:18.881550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96decc6-43c5-48d5-ace2-286a6f361b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:18.882404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96decc6-43c5-48d5-ace2-286a6f361b41 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:18.882683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96decc6-43c5-48d5-ace2-286a6f361b41 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:18.882954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96decc6-43c5-48d5-ace2-286a6f361b41 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:18.884193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96decc6-43c5-48d5-ace2-286a6f361b41 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:18.886075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96decc6-43c5-48d5-ace2-286a6f361b41 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:18.888750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 175/349] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:18 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:18.894229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96decc6-43c5-48d5-ace2-286a6f361b41 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:18.972361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e15819-1159-4b1a-b103-ee909b2d49b7 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:19.010199 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d771c-8ab7-460c-8442-e07cbac50589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:19.011202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d771c-8ab7-460c-8442-e07cbac50589 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.011489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d771c-8ab7-460c-8442-e07cbac50589 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.011764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d771c-8ab7-460c-8442-e07cbac50589 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.012605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d771c-8ab7-460c-8442-e07cbac50589 None admin] RBAC: Adding query filter params (group.id=3e6e4ab335da48048aced67d17c2c2ef, role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:19.012988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d771c-8ab7-460c-8442-e07cbac50589 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:19.014388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d771c-8ab7-460c-8442-e07cbac50589 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:19.025968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 175/350] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:18 2022] GET /identity/v3/role_assignments?group.id=3e6e4ab335da48048aced67d17c2c2ef&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:19.027145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:19.027958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31029-e2a4-4cde-8fb1-d5f8640dd0f8 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.027958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31029-e2a4-4cde-8fb1-d5f8640dd0f8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.028225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b31029-e2a4-4cde-8fb1-d5f8640dd0f8 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.029894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 176/351] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:19.030676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b31029-e2a4-4cde-8fb1-d5f8640dd0f8 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:19.036588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96decc6-43c5-48d5-ace2-286a6f361b41 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:19.079615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578ec34-6ebd-4e19-bf90-75e5545f27c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:19.081728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578ec34-6ebd-4e19-bf90-75e5545f27c9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.082242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578ec34-6ebd-4e19-bf90-75e5545f27c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.082701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578ec34-6ebd-4e19-bf90-75e5545f27c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.097177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578ec34-6ebd-4e19-bf90-75e5545f27c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-b578ec34-6ebd-4e19-bf90-75e5545f27c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:19.107057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:19.108082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:19.108907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:19.111974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 176/352] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:19.113704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b01e80-a031-42dc-a859-9f3e8ff19f1d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.114196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b01e80-a031-42dc-a859-9f3e8ff19f1d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.114670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b01e80-a031-42dc-a859-9f3e8ff19f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.134762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-73b01e80-a031-42dc-a859-9f3e8ff19f1d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:19.136363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-73b01e80-a031-42dc-a859-9f3e8ff19f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:19.187579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:19.212787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b31029-e2a4-4cde-8fb1-d5f8640dd0f8 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:19.231773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 177/353] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:19.232551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b01e80-a031-42dc-a859-9f3e8ff19f1d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:19.283271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af313d-28fc-4f3b-b93b-6a7b15389bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:19.284159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af313d-28fc-4f3b-b93b-6a7b15389bf6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.284437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af313d-28fc-4f3b-b93b-6a7b15389bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.284707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af313d-28fc-4f3b-b93b-6a7b15389bf6 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.286928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af313d-28fc-4f3b-b93b-6a7b15389bf6 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:19.288485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af313d-28fc-4f3b-b93b-6a7b15389bf6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:19.293918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 177/354] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:19 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:19.295484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40698b67-0755-49f8-b091-9e0be8addb75 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.295777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40698b67-0755-49f8-b091-9e0be8addb75 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.296056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40698b67-0755-49f8-b091-9e0be8addb75 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.297418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 178/355] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:19.300547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40698b67-0755-49f8-b091-9e0be8addb75 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:19.362228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:19.374486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b01e80-a031-42dc-a859-9f3e8ff19f1d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:19.416042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:19.416742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.417032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.417305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.418106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:19.418413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:19.420129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:19.431037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 178/356] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:19.431986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:19.439255 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40698b67-0755-49f8-b091-9e0be8addb75 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:19.484656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c9fca-378c-411e-980b-9e254601f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:19.485705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c9fca-378c-411e-980b-9e254601f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.486494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c9fca-378c-411e-980b-9e254601f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.486772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c9fca-378c-411e-980b-9e254601f5e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.501822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c9fca-378c-411e-980b-9e254601f5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-8b0c9fca-378c-411e-980b-9e254601f5e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:19.510704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:19.511979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:19.513454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:19.515105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 179/357] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:19.517167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dfe81-e946-4de5-aa22-0685dcd175d6 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.517167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dfe81-e946-4de5-aa22-0685dcd175d6 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.517167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dfe81-e946-4de5-aa22-0685dcd175d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.535202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e47dfe81-e946-4de5-aa22-0685dcd175d6 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:19.536398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e47dfe81-e946-4de5-aa22-0685dcd175d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:19.548554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:19.556728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3edb1-b588-4cf0-a026-2627a2048b16 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:19.609279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:19.610700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.610700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.610700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.611946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] RBAC: Adding query filter params (group.id=3e6e4ab335da48048aced67d17c2c2ef, role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:19.611946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:19.613152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:19.621863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 179/358] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:19 2022] GET /identity/v3/role_assignments?group.id=3e6e4ab335da48048aced67d17c2c2ef&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:19.623317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:19.640766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:19.642000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 180/359] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:19.643859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2d178-271d-4210-942a-cd11305c5f90 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.645964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2d178-271d-4210-942a-cd11305c5f90 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.646335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2d178-271d-4210-942a-cd11305c5f90 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.648350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 181/360] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:19.650282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2d178-271d-4210-942a-cd11305c5f90 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:19.782554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf514d45-637a-40bc-8ae8-3290f2f82928 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:19.810902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2d178-271d-4210-942a-cd11305c5f90 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:19.825814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbecf11-bb50-4bdb-9696-4a23056dd037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:19.826688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecf11-bb50-4bdb-9696-4a23056dd037 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.827026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecf11-bb50-4bdb-9696-4a23056dd037 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.827262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecf11-bb50-4bdb-9696-4a23056dd037 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.858952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1bfc7-8dc6-4ba9-ac4f-d9a8a719e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:19.859809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1bfc7-8dc6-4ba9-ac4f-d9a8a719e95b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.860102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1bfc7-8dc6-4ba9-ac4f-d9a8a719e95b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.860473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1bfc7-8dc6-4ba9-ac4f-d9a8a719e95b None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.861261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1bfc7-8dc6-4ba9-ac4f-d9a8a719e95b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:19.861566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1bfc7-8dc6-4ba9-ac4f-d9a8a719e95b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:19.863085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1bfc7-8dc6-4ba9-ac4f-d9a8a719e95b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:19.877924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 182/361] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:19.879338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79a6b5-eb0d-419e-9508-1b5e7f339ea5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:19.879632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79a6b5-eb0d-419e-9508-1b5e7f339ea5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:19.879915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79a6b5-eb0d-419e-9508-1b5e7f339ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:19.892220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:19.909714 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0c79a6b5-eb0d-419e-9508-1b5e7f339ea5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:19.911555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0c79a6b5-eb0d-419e-9508-1b5e7f339ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:19.917565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbecf11-bb50-4bdb-9696-4a23056dd037 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:19.919127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbecf11-bb50-4bdb-9696-4a23056dd037 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:19.937131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:20.012185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 183/362] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:20.013566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79a6b5-eb0d-419e-9508-1b5e7f339ea5 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:20.061878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 180/363] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:20.061878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbecf11-bb50-4bdb-9696-4a23056dd037 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:20.166087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79a6b5-eb0d-419e-9508-1b5e7f339ea5 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:20.199935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbecf11-bb50-4bdb-9696-4a23056dd037 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:20.211493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59892b-d434-4819-afb7-163db96ef677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:20.212320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59892b-d434-4819-afb7-163db96ef677 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:20.212604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59892b-d434-4819-afb7-163db96ef677 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:20.212942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59892b-d434-4819-afb7-163db96ef677 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:20.216161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:20.218624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:20.231638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59892b-d434-4819-afb7-163db96ef677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-ed59892b-d434-4819-afb7-163db96ef677 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:20.252778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:20.254195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 184/364] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:20.255522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59892b-d434-4819-afb7-163db96ef677 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:20.257912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae1ec9-4547-46eb-b172-e1382d61654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:20.259188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae1ec9-4547-46eb-b172-e1382d61654d None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:20.259677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae1ec9-4547-46eb-b172-e1382d61654d None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:20.260513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae1ec9-4547-46eb-b172-e1382d61654d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:20.356869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae1ec9-4547-46eb-b172-e1382d61654d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:20.359192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae1ec9-4547-46eb-b172-e1382d61654d None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:20.398511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59892b-d434-4819-afb7-163db96ef677 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:20.451491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b18b1f-6579-4f28-8452-b690a8575be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:20.452357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18b1f-6579-4f28-8452-b690a8575be8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:20.452641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18b1f-6579-4f28-8452-b690a8575be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:20.452923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18b1f-6579-4f28-8452-b690a8575be8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:20.476420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b18b1f-6579-4f28-8452-b690a8575be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-82b18b1f-6579-4f28-8452-b690a8575be8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:20.493329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:20.494699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 185/365] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:20.496024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b18b1f-6579-4f28-8452-b690a8575be8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:20.525218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 181/366] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:20.526970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae1ec9-4547-46eb-b172-e1382d61654d None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:20.545541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:20.594182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:20.630871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b18b1f-6579-4f28-8452-b690a8575be8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:20.644309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae1ec9-4547-46eb-b172-e1382d61654d None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:20.670236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a55f03-85a5-41dc-92fa-7b05978e7bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:20.671285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a55f03-85a5-41dc-92fa-7b05978e7bd7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:20.671716 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a55f03-85a5-41dc-92fa-7b05978e7bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:20.672006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a55f03-85a5-41dc-92fa-7b05978e7bd7 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:20.673331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a55f03-85a5-41dc-92fa-7b05978e7bd7 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:20.674984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a55f03-85a5-41dc-92fa-7b05978e7bd7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:20.677050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 186/367] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:20.677652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a55f03-85a5-41dc-92fa-7b05978e7bd7 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:20.688617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:20.689300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:20.689581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:20.690021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:20.690785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] RBAC: Adding query filter params (name=reader) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:20.691087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:20.692505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:20.702199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 182/368] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:20.703183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:20.782427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:20.818672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a55f03-85a5-41dc-92fa-7b05978e7bd7 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:20.823905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676dc1-e713-4361-8414-dba8a90614ed None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:20.861285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:20.864435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ead4e-0c85-45f8-9405-3b27603f1615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:20.865444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ead4e-0c85-45f8-9405-3b27603f1615 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:20.865731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ead4e-0c85-45f8-9405-3b27603f1615 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:20.866068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ead4e-0c85-45f8-9405-3b27603f1615 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:20.867271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ead4e-0c85-45f8-9405-3b27603f1615 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:20.868747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ead4e-0c85-45f8-9405-3b27603f1615 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:20.875334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 183/369] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:20.875334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ead4e-0c85-45f8-9405-3b27603f1615 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:20.882404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:20.882404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:20.883385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:20.883668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:20.884556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:20.884902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:20.887244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:20.896578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 187/370] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:20.897266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.002840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ead4e-0c85-45f8-9405-3b27603f1615 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.030273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c13a7-ce14-4156-bbcf-d26f4366e6c7 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.045435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9b38-7dca-4a01-b798-5fb07fa81095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.046933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9b38-7dca-4a01-b798-5fb07fa81095 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.047453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9b38-7dca-4a01-b798-5fb07fa81095 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.047984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9b38-7dca-4a01-b798-5fb07fa81095 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.049595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c9b38-7dca-4a01-b798-5fb07fa81095 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.051371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c9b38-7dca-4a01-b798-5fb07fa81095 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.057219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 184/371] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:21.058532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9b38-7dca-4a01-b798-5fb07fa81095 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.071279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b15d32-795e-4b53-96b5-c171de281708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.072202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b15d32-795e-4b53-96b5-c171de281708 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.073078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b15d32-795e-4b53-96b5-c171de281708 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.073078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b15d32-795e-4b53-96b5-c171de281708 None admin] PATH_INFO: `/v3/users/03bb8b8238a34e00ab0c6157785b914a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.074488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b15d32-795e-4b53-96b5-c171de281708 None admin] RBAC: Authorizing `identity:get_user(user_id=03bb8b8238a34e00ab0c6157785b914a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.075967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:21.076653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b15d32-795e-4b53-96b5-c171de281708 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.078771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 188/372] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:40:20 2022] GET /identity/v3/users/03bb8b8238a34e00ab0c6157785b914a => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:21.079571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b15d32-795e-4b53-96b5-c171de281708 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.172832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:21.184979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9b38-7dca-4a01-b798-5fb07fa81095 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.201024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b15d32-795e-4b53-96b5-c171de281708 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.230894 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401d04c-d97c-435f-89f9-c58097a1f291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.232935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401d04c-d97c-435f-89f9-c58097a1f291 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.233328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401d04c-d97c-435f-89f9-c58097a1f291 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.233710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401d04c-d97c-435f-89f9-c58097a1f291 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.235219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401d04c-d97c-435f-89f9-c58097a1f291 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.240145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401d04c-d97c-435f-89f9-c58097a1f291 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.246228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 185/373] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:21.247412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401d04c-d97c-435f-89f9-c58097a1f291 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.264719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed8b3c-b3c4-46b8-9f00-bd3d3c9adcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.266241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed8b3c-b3c4-46b8-9f00-bd3d3c9adcb8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.266754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed8b3c-b3c4-46b8-9f00-bd3d3c9adcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.268570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed8b3c-b3c4-46b8-9f00-bd3d3c9adcb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.359957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:21.364406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed8b3c-b3c4-46b8-9f00-bd3d3c9adcb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.366271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed8b3c-b3c4-46b8-9f00-bd3d3c9adcb8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.386924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401d04c-d97c-435f-89f9-c58097a1f291 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.422400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:21.425470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.425755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.426084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.426367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.427234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] RBAC: Adding query filter params (group.id=3e6e4ab335da48048aced67d17c2c2ef, role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=20a1163a83aa4705a89b4635d4e037b4) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:21.427550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.429023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.436704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 186/374] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/role_assignments?group.id=3e6e4ab335da48048aced67d17c2c2ef&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:21.437363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.479817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 189/375] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:21.480771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed8b3c-b3c4-46b8-9f00-bd3d3c9adcb8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.552328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822c498-4512-4332-8615-11b899ad4fe3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.591350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.592041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.592327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.592606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.593459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] RBAC: Adding query filter params (role.id=24a12434807d480fafb72632fc2d6756, user.id=03bb8b8238a34e00ab0c6157785b914a, scope.system=all) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:21.593793 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.595341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.596639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed8b3c-b3c4-46b8-9f00-bd3d3c9adcb8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.602076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:21.611784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 187/376] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/role_assignments?user.id=03bb8b8238a34e00ab0c6157785b914a&scope.system=all&role.id=24a12434807d480fafb72632fc2d6756 => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:21.612743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.637815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e9bf9-2818-426c-bbf7-d30b0692e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.638520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9bf9-2818-426c-bbf7-d30b0692e318 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.638806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9bf9-2818-426c-bbf7-d30b0692e318 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.639261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9bf9-2818-426c-bbf7-d30b0692e318 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.640326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e9bf9-2818-426c-bbf7-d30b0692e318 None admin] RBAC: Adding query filter params (group.id=3e6e4ab335da48048aced67d17c2c2ef, role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=20a1163a83aa4705a89b4635d4e037b4) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:21.640638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e9bf9-2818-426c-bbf7-d30b0692e318 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.642202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e9bf9-2818-426c-bbf7-d30b0692e318 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.648941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 190/377] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/role_assignments?group.id=3e6e4ab335da48048aced67d17c2c2ef&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:21.723485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f556-8fb8-4139-9f3a-72dc5729d8eb None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.723841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f556-8fb8-4139-9f3a-72dc5729d8eb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.724035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f556-8fb8-4139-9f3a-72dc5729d8eb None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.728296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 191/378] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:40:21.742041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c3082-6eb2-491f-ac63-3637d6c8c9dd None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.748489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb9649-3805-445e-9694-1d08f0f60921 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.748851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb9649-3805-445e-9694-1d08f0f60921 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.749007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb9649-3805-445e-9694-1d08f0f60921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.782797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-46eb9649-3805-445e-9694-1d08f0f60921 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:21.784098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7943b-868a-483a-85c8-ec9b3f420899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.784296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-46eb9649-3805-445e-9694-1d08f0f60921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:21.785065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7943b-868a-483a-85c8-ec9b3f420899 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.785337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7943b-868a-483a-85c8-ec9b3f420899 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.785613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7943b-868a-483a-85c8-ec9b3f420899 None admin] PATH_INFO: `/v3/projects/eb46a2622cdd481281f00889cb2ab74f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.787031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7943b-868a-483a-85c8-ec9b3f420899 None admin] RBAC: Authorizing `identity:get_project(project_id=eb46a2622cdd481281f00889cb2ab74f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.788599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7943b-868a-483a-85c8-ec9b3f420899 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.790924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 188/379] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/projects/eb46a2622cdd481281f00889cb2ab74f => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:21.798266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7943b-868a-483a-85c8-ec9b3f420899 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.805391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:21.884564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:21.900935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 192/380] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:21.919735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7943b-868a-483a-85c8-ec9b3f420899 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:21.951727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb9649-3805-445e-9694-1d08f0f60921 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:21.957827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43263407-912d-4a7f-bd8f-55cddf6a50e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:21.958720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43263407-912d-4a7f-bd8f-55cddf6a50e4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:21.959004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43263407-912d-4a7f-bd8f-55cddf6a50e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:21.959287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43263407-912d-4a7f-bd8f-55cddf6a50e4 None admin] PATH_INFO: `/v3/groups/1a37c2d4d7da41808bc8e8f7d080e53c` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:21.968385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43263407-912d-4a7f-bd8f-55cddf6a50e4 None admin] RBAC: Authorizing `identity:get_group(group_id=1a37c2d4d7da41808bc8e8f7d080e53c)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:21.969900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43263407-912d-4a7f-bd8f-55cddf6a50e4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:21.972042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 189/381] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/groups/1a37c2d4d7da41808bc8e8f7d080e53c => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:21.977793 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43263407-912d-4a7f-bd8f-55cddf6a50e4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:22.075503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb9649-3805-445e-9694-1d08f0f60921 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:22.088351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43263407-912d-4a7f-bd8f-55cddf6a50e4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:22.113949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca97e1-2ec0-4d92-8991-ed9d68290046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:22.114941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca97e1-2ec0-4d92-8991-ed9d68290046 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:22.115222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca97e1-2ec0-4d92-8991-ed9d68290046 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:22.115492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca97e1-2ec0-4d92-8991-ed9d68290046 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:22.122186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:22.127406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca97e1-2ec0-4d92-8991-ed9d68290046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:22.134586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:22.135505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:22.135995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:22.136344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-c0ca97e1-2ec0-4d92-8991-ed9d68290046 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:22.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:22.137676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:22.138736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:22.139332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] RBAC: Adding query filter params (group.id=1a37c2d4d7da41808bc8e8f7d080e53c, role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=eb46a2622cdd481281f00889cb2ab74f) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:22.139332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:22.139449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 193/382] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:22.139525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:22.145813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca97e1-2ec0-4d92-8991-ed9d68290046 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:22.150836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 190/383] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:21 2022] GET /identity/v3/role_assignments?group.id=1a37c2d4d7da41808bc8e8f7d080e53c&scope.project.id=eb46a2622cdd481281f00889cb2ab74f&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:22.157708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:22.300228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca97e1-2ec0-4d92-8991-ed9d68290046 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:22.354527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082da41d-20fe-4ecc-86e6-c2e67d82f907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:22.355229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da41d-20fe-4ecc-86e6-c2e67d82f907 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:22.355510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da41d-20fe-4ecc-86e6-c2e67d82f907 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:22.355779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da41d-20fe-4ecc-86e6-c2e67d82f907 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:22.356563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082da41d-20fe-4ecc-86e6-c2e67d82f907 None admin] RBAC: Adding query filter params (name=member) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:22.356869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082da41d-20fe-4ecc-86e6-c2e67d82f907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:22.358324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082da41d-20fe-4ecc-86e6-c2e67d82f907 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:22.368455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 194/384] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:22.374563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:22.503698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774f7d5-879f-45ff-8d57-9e96fa1550ce None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:22.530268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46935240-38fb-4da3-bfbd-bb948bb37d23 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:22.530709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46935240-38fb-4da3-bfbd-bb948bb37d23 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:22.531086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46935240-38fb-4da3-bfbd-bb948bb37d23 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:22.532845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 195/385] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:40:22.538765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce80284-b3af-405e-9815-f93620a5f478 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:22.539174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce80284-b3af-405e-9815-f93620a5f478 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:22.539546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce80284-b3af-405e-9815-f93620a5f478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:22.547797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5968f-bae7-49ff-81a3-23c9916b27e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:22.548954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5968f-bae7-49ff-81a3-23c9916b27e1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:22.549370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5968f-bae7-49ff-81a3-23c9916b27e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:22.549856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5968f-bae7-49ff-81a3-23c9916b27e1 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:22.551192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5968f-bae7-49ff-81a3-23c9916b27e1 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:22.552779 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5968f-bae7-49ff-81a3-23c9916b27e1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:22.554952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 191/386] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:22 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:22.561370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5968f-bae7-49ff-81a3-23c9916b27e1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:22.574860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7ce80284-b3af-405e-9815-f93620a5f478 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:22.576231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7ce80284-b3af-405e-9815-f93620a5f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:22.645695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:22.670589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:22.676402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 196/387] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:22.715636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5968f-bae7-49ff-81a3-23c9916b27e1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:22.719355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce80284-b3af-405e-9815-f93620a5f478 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:22.778619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d451a-109e-482a-b32d-ba1d21125ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:22.779918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d451a-109e-482a-b32d-ba1d21125ea4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:22.781102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d451a-109e-482a-b32d-ba1d21125ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:22.781713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d451a-109e-482a-b32d-ba1d21125ea4 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:22.784937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d451a-109e-482a-b32d-ba1d21125ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:22.786798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d451a-109e-482a-b32d-ba1d21125ea4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:22.789284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 192/388] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:22 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:22.798537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d451a-109e-482a-b32d-ba1d21125ea4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:22.928935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d451a-109e-482a-b32d-ba1d21125ea4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:22.966737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696a117-1b17-485d-8de3-7de3403f4281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:22.967661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696a117-1b17-485d-8de3-7de3403f4281 None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:22.967947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696a117-1b17-485d-8de3-7de3403f4281 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:22.968232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696a117-1b17-485d-8de3-7de3403f4281 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/groups/3e6e4ab335da48048aced67d17c2c2ef/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:22.970353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696a117-1b17-485d-8de3-7de3403f4281 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a4deb4f0ae34b60b855c91ae5f02d62, group_id=3e6e4ab335da48048aced67d17c2c2ef, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:22.971738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce80284-b3af-405e-9815-f93620a5f478 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:22.971850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696a117-1b17-485d-8de3-7de3403f4281 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:22.974226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:23.001560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 193/389] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jun 21 15:40:22 2022] PUT /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/groups/3e6e4ab335da48048aced67d17c2c2ef/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:23.031856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8fe9e-031f-412e-8697-2e64ccbcb3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:23.032627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8fe9e-031f-412e-8697-2e64ccbcb3c5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:23.032909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8fe9e-031f-412e-8697-2e64ccbcb3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:23.033183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8fe9e-031f-412e-8697-2e64ccbcb3c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:23.056011 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8fe9e-031f-412e-8697-2e64ccbcb3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f2a8fe9e-031f-412e-8697-2e64ccbcb3c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:23.069149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:23.070168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:23.070957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 197/390] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:23.078374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696a117-1b17-485d-8de3-7de3403f4281 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:23.200308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696a117-1b17-485d-8de3-7de3403f4281 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:23.251756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:23.252467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:23.252837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:23.253034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:23.253826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:23.254138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:23.255542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:23.268336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 194/391] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:23.274478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8fe9e-031f-412e-8697-2e64ccbcb3c5 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:23.351631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:23.404169 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8fe9e-031f-412e-8697-2e64ccbcb3c5 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:23.445697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73bf13-1663-46ec-8869-1ca2c2f97def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:23.446612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73bf13-1663-46ec-8869-1ca2c2f97def None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:23.446890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73bf13-1663-46ec-8869-1ca2c2f97def None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:23.447196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73bf13-1663-46ec-8869-1ca2c2f97def None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:23.448956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73bf13-1663-46ec-8869-1ca2c2f97def None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:23.450527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73bf13-1663-46ec-8869-1ca2c2f97def None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:23.452470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 198/392] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:23 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:23.460353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:23.519573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:23.577918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b84d42-0d59-4336-aa1d-6158d2f41e1b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:23.614367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a467df9-174d-43fe-a1da-cb7e8420e9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:23.615169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a467df9-174d-43fe-a1da-cb7e8420e9c8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:23.615449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a467df9-174d-43fe-a1da-cb7e8420e9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:23.615718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a467df9-174d-43fe-a1da-cb7e8420e9c8 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:23.616943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a467df9-174d-43fe-a1da-cb7e8420e9c8 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:23.618440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a467df9-174d-43fe-a1da-cb7e8420e9c8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:23.620644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 195/393] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:23 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:23.626242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73bf13-1663-46ec-8869-1ca2c2f97def None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:23.734448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73bf13-1663-46ec-8869-1ca2c2f97def None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:23.761234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:23.770558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282696b1-30b8-4b8b-add7-ac07ae8e0262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:23.771466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282696b1-30b8-4b8b-add7-ac07ae8e0262 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:23.771755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282696b1-30b8-4b8b-add7-ac07ae8e0262 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:23.772034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282696b1-30b8-4b8b-add7-ac07ae8e0262 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/groups/3e6e4ab335da48048aced67d17c2c2ef/roles/a505b2a721044e6ea8312ef6ffdc1c9f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:23.774656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282696b1-30b8-4b8b-add7-ac07ae8e0262 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a4deb4f0ae34b60b855c91ae5f02d62, group_id=3e6e4ab335da48048aced67d17c2c2ef, role_id=a505b2a721044e6ea8312ef6ffdc1c9f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:23.776542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282696b1-30b8-4b8b-add7-ac07ae8e0262 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:23.793315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 199/394] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jun 21 15:40:23 2022] PUT /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62/groups/3e6e4ab335da48048aced67d17c2c2ef/roles/a505b2a721044e6ea8312ef6ffdc1c9f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:24.472130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c780bd-ae9b-42ea-85e8-7468e5a2d463 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:24.473141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c780bd-ae9b-42ea-85e8-7468e5a2d463 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:24.473648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c780bd-ae9b-42ea-85e8-7468e5a2d463 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:24.475600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 196/395] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:24.481518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae0f04-238f-4490-9385-8255c5c0b200 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:24.482435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae0f04-238f-4490-9385-8255c5c0b200 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:24.482940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae0f04-238f-4490-9385-8255c5c0b200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:24.505336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-42ae0f04-238f-4490-9385-8255c5c0b200 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:24.507027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-42ae0f04-238f-4490-9385-8255c5c0b200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:24.599009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 200/396] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:24.618575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c6e7-ed1d-4d74-b62e-8d4c2e694751 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:24.618922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c6e7-ed1d-4d74-b62e-8d4c2e694751 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:24.619272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c6e7-ed1d-4d74-b62e-8d4c2e694751 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:24.622393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 197/397] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:24.627144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea00f1-a3e0-4090-933a-58d4addb8ff4 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:24.627526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea00f1-a3e0-4090-933a-58d4addb8ff4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:24.627722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea00f1-a3e0-4090-933a-58d4addb8ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:24.629359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3c6e7-ed1d-4d74-b62e-8d4c2e694751 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:24.650550 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-16ea00f1-a3e0-4090-933a-58d4addb8ff4 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:24.652307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-16ea00f1-a3e0-4090-933a-58d4addb8ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:24.754113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 201/398] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:24.794829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea00f1-a3e0-4090-933a-58d4addb8ff4 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:24.813655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3c6e7-ed1d-4d74-b62e-8d4c2e694751 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:24.854609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:24.886228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:24.891534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeabb27-a529-43bc-b611-69e7d567f979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:24.892654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeabb27-a529-43bc-b611-69e7d567f979 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:24.893135 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeabb27-a529-43bc-b611-69e7d567f979 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:24.893639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeabb27-a529-43bc-b611-69e7d567f979 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:24.916770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeabb27-a529-43bc-b611-69e7d567f979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-3aeabb27-a529-43bc-b611-69e7d567f979 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:24.931074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:24.932952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:24.933985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:24.936967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 198/399] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:24.949764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea00f1-a3e0-4090-933a-58d4addb8ff4 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:24.952532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeabb27-a529-43bc-b611-69e7d567f979 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:24.988467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef90391-9ecb-409c-8744-4342ced668bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:24.989338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef90391-9ecb-409c-8744-4342ced668bb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:24.989620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef90391-9ecb-409c-8744-4342ced668bb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:24.990998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef90391-9ecb-409c-8744-4342ced668bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.005811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef90391-9ecb-409c-8744-4342ced668bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-2ef90391-9ecb-409c-8744-4342ced668bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:25.010712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:25.011590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:25.012351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:25.016893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 202/400] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:25.018207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef90391-9ecb-409c-8744-4342ced668bb None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.127904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeabb27-a529-43bc-b611-69e7d567f979 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.171307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef90391-9ecb-409c-8744-4342ced668bb None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.176908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820004ad-3539-4f0d-a8c5-b107dac0b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.177605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820004ad-3539-4f0d-a8c5-b107dac0b930 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.177935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820004ad-3539-4f0d-a8c5-b107dac0b930 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.178213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820004ad-3539-4f0d-a8c5-b107dac0b930 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.178978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820004ad-3539-4f0d-a8c5-b107dac0b930 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:25.179718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:25.180416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820004ad-3539-4f0d-a8c5-b107dac0b930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:25.182560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820004ad-3539-4f0d-a8c5-b107dac0b930 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:25.193656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 199/401] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:25.195072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070622c9-18b3-4c0e-863d-1882403dfe6a None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.195362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070622c9-18b3-4c0e-863d-1882403dfe6a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.195640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070622c9-18b3-4c0e-863d-1882403dfe6a None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.202257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 200/402] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:25.204464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070622c9-18b3-4c0e-863d-1882403dfe6a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.223374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6474b-5d57-4c86-8adc-21b55f38aeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.224231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6474b-5d57-4c86-8adc-21b55f38aeca None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.224518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6474b-5d57-4c86-8adc-21b55f38aeca None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.224800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6474b-5d57-4c86-8adc-21b55f38aeca None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.225568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6474b-5d57-4c86-8adc-21b55f38aeca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:25.225901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6474b-5d57-4c86-8adc-21b55f38aeca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:25.227592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6474b-5d57-4c86-8adc-21b55f38aeca None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:25.252856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 203/403] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:25.252856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cc4a7-baa2-4fda-b767-2426d1f00f44 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.252856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cc4a7-baa2-4fda-b767-2426d1f00f44 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.252856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cc4a7-baa2-4fda-b767-2426d1f00f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.268719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:25.274130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e38cc4a7-baa2-4fda-b767-2426d1f00f44 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:25.275651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e38cc4a7-baa2-4fda-b767-2426d1f00f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:25.333853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070622c9-18b3-4c0e-863d-1882403dfe6a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.358078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 204/404] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:25.359170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cc4a7-baa2-4fda-b767-2426d1f00f44 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.369490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b0707-18d3-4b66-a2ca-b268e4b45605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.370732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b0707-18d3-4b66-a2ca-b268e4b45605 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.371226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b0707-18d3-4b66-a2ca-b268e4b45605 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.371685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b0707-18d3-4b66-a2ca-b268e4b45605 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.373070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b0707-18d3-4b66-a2ca-b268e4b45605 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:25.374961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b0707-18d3-4b66-a2ca-b268e4b45605 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:25.381347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 201/405] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:25 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:25.382820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b0707-18d3-4b66-a2ca-b268e4b45605 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.479337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:25.487384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cc4a7-baa2-4fda-b767-2426d1f00f44 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.500644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b0707-18d3-4b66-a2ca-b268e4b45605 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.515115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:25.527078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d2db9-4bbc-4a52-842f-f4d772c5f93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.528290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2db9-4bbc-4a52-842f-f4d772c5f93f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.528759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2db9-4bbc-4a52-842f-f4d772c5f93f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.529228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d2db9-4bbc-4a52-842f-f4d772c5f93f None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.530527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d2db9-4bbc-4a52-842f-f4d772c5f93f None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:25.532034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d2db9-4bbc-4a52-842f-f4d772c5f93f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:25.538089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 205/406] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:25 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:25.538089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d2db9-4bbc-4a52-842f-f4d772c5f93f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.550060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d52d68-fdfe-45e4-8cc5-db69e7e1e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.551014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d52d68-fdfe-45e4-8cc5-db69e7e1e971 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.551359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d52d68-fdfe-45e4-8cc5-db69e7e1e971 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.551698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d52d68-fdfe-45e4-8cc5-db69e7e1e971 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.552943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d52d68-fdfe-45e4-8cc5-db69e7e1e971 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:25.554580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d52d68-fdfe-45e4-8cc5-db69e7e1e971 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:25.561803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 202/407] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:25 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:25.562688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d52d68-fdfe-45e4-8cc5-db69e7e1e971 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.669371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d2db9-4bbc-4a52-842f-f4d772c5f93f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.689465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d52d68-fdfe-45e4-8cc5-db69e7e1e971 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.718268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5185e25-062c-494d-9cac-19d600610ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.719349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185e25-062c-494d-9cac-19d600610ba3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.720005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185e25-062c-494d-9cac-19d600610ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.720970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185e25-062c-494d-9cac-19d600610ba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.735976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5185e25-062c-494d-9cac-19d600610ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:25.736301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea47a2e-6baa-4d57-9c7f-c597469acb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.737282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea47a2e-6baa-4d57-9c7f-c597469acb1f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.737664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea47a2e-6baa-4d57-9c7f-c597469acb1f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.738060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea47a2e-6baa-4d57-9c7f-c597469acb1f None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.739453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea47a2e-6baa-4d57-9c7f-c597469acb1f None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:25.741056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea47a2e-6baa-4d57-9c7f-c597469acb1f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:25.747777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 203/408] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:25 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:25.747777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea47a2e-6baa-4d57-9c7f-c597469acb1f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-d5185e25-062c-494d-9cac-19d600610ba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:25.748435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:25.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:25.750071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 206/409] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:25.750708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5185e25-062c-494d-9cac-19d600610ba3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.789682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:25.807418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:25.866028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea47a2e-6baa-4d57-9c7f-c597469acb1f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.875776 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5185e25-062c-494d-9cac-19d600610ba3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:25.905206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f902400-4491-4a24-9424-4f0049ee04e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.906130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902400-4491-4a24-9424-4f0049ee04e8 None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.906409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902400-4491-4a24-9424-4f0049ee04e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.906686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f902400-4491-4a24-9424-4f0049ee04e8 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4/groups/3e6e4ab335da48048aced67d17c2c2ef/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.908682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f902400-4491-4a24-9424-4f0049ee04e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=20a1163a83aa4705a89b4635d4e037b4, group_id=3e6e4ab335da48048aced67d17c2c2ef, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:25.910163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f902400-4491-4a24-9424-4f0049ee04e8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:25.911925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cb7a2-dd2c-417d-8f82-6ed0d88b1b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:25.913739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cb7a2-dd2c-417d-8f82-6ed0d88b1b75 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.914114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cb7a2-dd2c-417d-8f82-6ed0d88b1b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.914344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cb7a2-dd2c-417d-8f82-6ed0d88b1b75 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4/groups/3e6e4ab335da48048aced67d17c2c2ef/roles/a505b2a721044e6ea8312ef6ffdc1c9f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.916797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cb7a2-dd2c-417d-8f82-6ed0d88b1b75 None admin] RBAC: Authorizing `identity:create_grant(project_id=20a1163a83aa4705a89b4635d4e037b4, group_id=3e6e4ab335da48048aced67d17c2c2ef, role_id=a505b2a721044e6ea8312ef6ffdc1c9f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:25.918361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cb7a2-dd2c-417d-8f82-6ed0d88b1b75 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:25.938764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 207/411] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jun 21 15:40:25 2022] PUT /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4/groups/3e6e4ab335da48048aced67d17c2c2ef/roles/a505b2a721044e6ea8312ef6ffdc1c9f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:25.938764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cb7a2-dd2c-417d-8f82-6ed0d88b1b75 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:25.939008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 204/411] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jun 21 15:40:25 2022] PUT /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4/groups/3e6e4ab335da48048aced67d17c2c2ef/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:25.939008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccf923-5327-4725-9b46-232120f2c016 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.939008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccf923-5327-4725-9b46-232120f2c016 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.939008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccf923-5327-4725-9b46-232120f2c016 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.948546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 205/412] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:25.949332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbcf25-14c2-4efb-9d00-5add5db05bd0 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:25.949624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbcf25-14c2-4efb-9d00-5add5db05bd0 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:25.949996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbcf25-14c2-4efb-9d00-5add5db05bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:25.988590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-52fbcf25-14c2-4efb-9d00-5add5db05bd0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:25.989870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-52fbcf25-14c2-4efb-9d00-5add5db05bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:26.115764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:26.129058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cb7a2-dd2c-417d-8f82-6ed0d88b1b75 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:26.129742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 206/413] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:26.156182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbcf25-14c2-4efb-9d00-5add5db05bd0 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:26.177381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:26.178457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.178752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.179027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.179958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:26.180564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:26.182606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:26.193485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 208/414] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:26.199522 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:26.201303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:26.284247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbcf25-14c2-4efb-9d00-5add5db05bd0 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:26.346041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5656d89-4877-41e8-ab29-1f14fb06dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:26.347347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5656d89-4877-41e8-ab29-1f14fb06dec3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.347851 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5656d89-4877-41e8-ab29-1f14fb06dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.348321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5656d89-4877-41e8-ab29-1f14fb06dec3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.354387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859a367-56cd-471d-a85e-0634ccf48f1a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:26.376375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5656d89-4877-41e8-ab29-1f14fb06dec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-d5656d89-4877-41e8-ab29-1f14fb06dec3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:26.384266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:26.385227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:26.386171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 207/415] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:26.393180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5656d89-4877-41e8-ab29-1f14fb06dec3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:26.436496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f2827-25ec-4672-a117-b166af78c160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:26.437766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f2827-25ec-4672-a117-b166af78c160 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.438264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f2827-25ec-4672-a117-b166af78c160 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.438728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f2827-25ec-4672-a117-b166af78c160 None admin] PATH_INFO: `/v3/groups/1a37c2d4d7da41808bc8e8f7d080e53c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.440242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f2827-25ec-4672-a117-b166af78c160 None admin] RBAC: Authorizing `identity:get_group(group_id=1a37c2d4d7da41808bc8e8f7d080e53c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:26.441922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f2827-25ec-4672-a117-b166af78c160 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:26.447725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 209/416] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:26 2022] GET /identity/v3/groups/1a37c2d4d7da41808bc8e8f7d080e53c => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:26.450776 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f2827-25ec-4672-a117-b166af78c160 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:26.481410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:26.586808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:26.597613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5656d89-4877-41e8-ab29-1f14fb06dec3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:26.624681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f2827-25ec-4672-a117-b166af78c160 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:26.633222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea40e9-cba9-48a7-8f12-98f89f926199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:26.633939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea40e9-cba9-48a7-8f12-98f89f926199 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.634221 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea40e9-cba9-48a7-8f12-98f89f926199 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.634525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea40e9-cba9-48a7-8f12-98f89f926199 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.635278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea40e9-cba9-48a7-8f12-98f89f926199 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:26.635583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea40e9-cba9-48a7-8f12-98f89f926199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:26.637063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea40e9-cba9-48a7-8f12-98f89f926199 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:26.646689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 208/417] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:26.652784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f41b69-b76e-40d5-bd92-cf611f24a167 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.652784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f41b69-b76e-40d5-bd92-cf611f24a167 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.652784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f41b69-b76e-40d5-bd92-cf611f24a167 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.654610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 209/418] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:26.655559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f41b69-b76e-40d5-bd92-cf611f24a167 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:26.669401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31956e0-1a84-47b4-9814-b109050ac55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:26.670690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31956e0-1a84-47b4-9814-b109050ac55d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.671241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31956e0-1a84-47b4-9814-b109050ac55d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.671709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31956e0-1a84-47b4-9814-b109050ac55d None admin] PATH_INFO: `/v3/projects/eb46a2622cdd481281f00889cb2ab74f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.673205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31956e0-1a84-47b4-9814-b109050ac55d None admin] RBAC: Authorizing `identity:get_project(project_id=eb46a2622cdd481281f00889cb2ab74f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:26.674972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31956e0-1a84-47b4-9814-b109050ac55d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:26.677286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 210/419] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:26 2022] GET /identity/v3/projects/eb46a2622cdd481281f00889cb2ab74f => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:26.678689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c763a23-a5e3-4bc1-bb3f-1bf40a49f568 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.679182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c763a23-a5e3-4bc1-bb3f-1bf40a49f568 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.679648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c763a23-a5e3-4bc1-bb3f-1bf40a49f568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.698181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3c763a23-a5e3-4bc1-bb3f-1bf40a49f568 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:26.699789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3c763a23-a5e3-4bc1-bb3f-1bf40a49f568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:26.767918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f41b69-b76e-40d5-bd92-cf611f24a167 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:26.781611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 211/420] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:26.781611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c763a23-a5e3-4bc1-bb3f-1bf40a49f568 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:26.807243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:26.821664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c77916-e8fb-4f71-8db2-5eee1fdf37fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:26.822603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77916-e8fb-4f71-8db2-5eee1fdf37fe None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.822896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77916-e8fb-4f71-8db2-5eee1fdf37fe None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.823301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77916-e8fb-4f71-8db2-5eee1fdf37fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.853887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:26.905733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c77916-e8fb-4f71-8db2-5eee1fdf37fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:26.908353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c77916-e8fb-4f71-8db2-5eee1fdf37fe None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:26.923115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c763a23-a5e3-4bc1-bb3f-1bf40a49f568 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:26.964118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be3d55-3f4f-48d2-b095-fe87ceca59c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:26.964924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3d55-3f4f-48d2-b095-fe87ceca59c1 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:26.965203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3d55-3f4f-48d2-b095-fe87ceca59c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:26.965549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be3d55-3f4f-48d2-b095-fe87ceca59c1 None admin] PATH_INFO: `/v3/projects/eb46a2622cdd481281f00889cb2ab74f/groups/1a37c2d4d7da41808bc8e8f7d080e53c/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:26.967930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be3d55-3f4f-48d2-b095-fe87ceca59c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb46a2622cdd481281f00889cb2ab74f, group_id=1a37c2d4d7da41808bc8e8f7d080e53c, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:26.969440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be3d55-3f4f-48d2-b095-fe87ceca59c1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:26.984645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 212/421] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Jun 21 15:40:26 2022] PUT /identity/v3/projects/eb46a2622cdd481281f00889cb2ab74f/groups/1a37c2d4d7da41808bc8e8f7d080e53c/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:26.985610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be3d55-3f4f-48d2-b095-fe87ceca59c1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:27.032500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 210/422] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:27.041140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c77916-e8fb-4f71-8db2-5eee1fdf37fe None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:27.083752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:27.120474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be3d55-3f4f-48d2-b095-fe87ceca59c1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:27.161723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dc2f6-4743-41eb-95e6-f7ffc7aa62ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:27.162599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dc2f6-4743-41eb-95e6-f7ffc7aa62ab None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:27.163328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dc2f6-4743-41eb-95e6-f7ffc7aa62ab None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:27.163328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dc2f6-4743-41eb-95e6-f7ffc7aa62ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:27.176275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1dc2f6-4743-41eb-95e6-f7ffc7aa62ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-2c1dc2f6-4743-41eb-95e6-f7ffc7aa62ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:27.189152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:27.190103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:27.190874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:27.191982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 213/423] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:27.194757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:27.197976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dc2f6-4743-41eb-95e6-f7ffc7aa62ab None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:27.251671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c77916-e8fb-4f71-8db2-5eee1fdf37fe None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:27.291542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357146db-57fa-4cdb-a55e-748e22ce89ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:27.292543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357146db-57fa-4cdb-a55e-748e22ce89ea None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:27.293101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357146db-57fa-4cdb-a55e-748e22ce89ea None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:27.293487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357146db-57fa-4cdb-a55e-748e22ce89ea None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:27.296026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357146db-57fa-4cdb-a55e-748e22ce89ea None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:27.297733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357146db-57fa-4cdb-a55e-748e22ce89ea None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:27.299952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 211/424] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:27 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:27.305658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357146db-57fa-4cdb-a55e-748e22ce89ea None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:27.330225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dc2f6-4743-41eb-95e6-f7ffc7aa62ab None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:27.398659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:27.399370 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:27.399659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:27.399935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:27.400943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:27.401261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:27.402721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:27.416094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 214/425] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:27.416387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357146db-57fa-4cdb-a55e-748e22ce89ea None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:27.416525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:27.421545 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:27.447360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:27.458617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf0995-3c14-4bc4-8629-c593c8ff92d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:27.459553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf0995-3c14-4bc4-8629-c593c8ff92d2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:27.459874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf0995-3c14-4bc4-8629-c593c8ff92d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:27.460154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf0995-3c14-4bc4-8629-c593c8ff92d2 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:27.461424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf0995-3c14-4bc4-8629-c593c8ff92d2 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:27.463333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf0995-3c14-4bc4-8629-c593c8ff92d2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:27.465360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 212/426] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:27 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:27.471193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf0995-3c14-4bc4-8629-c593c8ff92d2 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:27.578349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf0995-3c14-4bc4-8629-c593c8ff92d2 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:27.614958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:27.615756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:27.615917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:27.616160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:27.617024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] RBAC: Adding query filter params (group.id=3e6e4ab335da48048aced67d17c2c2ef, role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:27.617337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:27.619381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:27.626237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 213/427] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:27 2022] GET /identity/v3/role_assignments?group.id=3e6e4ab335da48048aced67d17c2c2ef&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:27.629824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5c5b5-8eb1-4eae-b4b4-e5b50088402f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:27.693580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4bf8-db40-496a-b091-d6f69bd75047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:27.694667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4bf8-db40-496a-b091-d6f69bd75047 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:27.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4bf8-db40-496a-b091-d6f69bd75047 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:27.695625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4bf8-db40-496a-b091-d6f69bd75047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:27.782606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:27.790524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4bf8-db40-496a-b091-d6f69bd75047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:27.792540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4bf8-db40-496a-b091-d6f69bd75047 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:27.913185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 215/428] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 15:40:27.918574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:27.988499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:28.036849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055d5b2-fb62-4c7b-8147-932db89fb633 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:28.081966 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf6664-bd4e-455f-bef4-8864ad090f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:28.082866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf6664-bd4e-455f-bef4-8864ad090f46 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.083155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf6664-bd4e-455f-bef4-8864ad090f46 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.083431 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf6664-bd4e-455f-bef4-8864ad090f46 None admin] PATH_INFO: `/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.084656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf6664-bd4e-455f-bef4-8864ad090f46 None admin] RBAC: Authorizing `identity:get_project(project_id=0a4deb4f0ae34b60b855c91ae5f02d62)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:28.086173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf6664-bd4e-455f-bef4-8864ad090f46 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:28.088347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 214/429] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:27 2022] GET /identity/v3/projects/0a4deb4f0ae34b60b855c91ae5f02d62 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:28.093991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4bf8-db40-496a-b091-d6f69bd75047 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:28.206298 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:28.214189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4bf8-db40-496a-b091-d6f69bd75047 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:28.275427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97553daa-a370-45e5-9077-780509d10080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:28.276734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97553daa-a370-45e5-9077-780509d10080 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.277319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97553daa-a370-45e5-9077-780509d10080 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.277813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97553daa-a370-45e5-9077-780509d10080 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.281163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97553daa-a370-45e5-9077-780509d10080 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:28.282990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97553daa-a370-45e5-9077-780509d10080 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:28.285352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 216/430] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:28 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:28.291089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf6664-bd4e-455f-bef4-8864ad090f46 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:28.399272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06253e5-1756-466d-8298-d7885dd9cc3f None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.399541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06253e5-1756-466d-8298-d7885dd9cc3f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.399823 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06253e5-1756-466d-8298-d7885dd9cc3f None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.402048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 217/431] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:28.407878 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9843d-163c-4402-be00-679436b995d7 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.408198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9843d-163c-4402-be00-679436b995d7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.408544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9843d-163c-4402-be00-679436b995d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.410615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf6664-bd4e-455f-bef4-8864ad090f46 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:28.429485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-5ee9843d-163c-4402-be00-679436b995d7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:28.431655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5ee9843d-163c-4402-be00-679436b995d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:28.440055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:28.450912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249a587-9b05-4c01-896c-1d2f7a2405eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:28.451597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a587-9b05-4c01-896c-1d2f7a2405eb None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.451882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a587-9b05-4c01-896c-1d2f7a2405eb None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.452162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249a587-9b05-4c01-896c-1d2f7a2405eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.453014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249a587-9b05-4c01-896c-1d2f7a2405eb None admin] RBAC: Adding query filter params (group.id=3e6e4ab335da48048aced67d17c2c2ef, role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:28.453329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249a587-9b05-4c01-896c-1d2f7a2405eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:28.454834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249a587-9b05-4c01-896c-1d2f7a2405eb None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:28.461575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 215/432] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:28 2022] GET /identity/v3/role_assignments?group.id=3e6e4ab335da48048aced67d17c2c2ef&scope.project.id=0a4deb4f0ae34b60b855c91ae5f02d62&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:28.527266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 218/433] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:28.534238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47a9fb-a942-4a6f-b3dd-1173fd73901c None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.534539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47a9fb-a942-4a6f-b3dd-1173fd73901c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.534822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47a9fb-a942-4a6f-b3dd-1173fd73901c None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.536452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 216/434] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:28.546300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a9700-b961-4b25-b464-c609c7cfa6bd None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.546958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a9700-b961-4b25-b464-c609c7cfa6bd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.547430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a9700-b961-4b25-b464-c609c7cfa6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.554260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47a9fb-a942-4a6f-b3dd-1173fd73901c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:28.567992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-689a9700-b961-4b25-b464-c609c7cfa6bd None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:28.569553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-689a9700-b961-4b25-b464-c609c7cfa6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:28.659268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 219/435] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:28.683280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a9700-b961-4b25-b464-c609c7cfa6bd None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:28.695845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47a9fb-a942-4a6f-b3dd-1173fd73901c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:28.709134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:28.744822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6ed35-4d1d-4c52-aff7-6851b908a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:28.745562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6ed35-4d1d-4c52-aff7-6851b908a8ff None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.745957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6ed35-4d1d-4c52-aff7-6851b908a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.746232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6ed35-4d1d-4c52-aff7-6851b908a8ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.758878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6ed35-4d1d-4c52-aff7-6851b908a8ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-3ff6ed35-4d1d-4c52-aff7-6851b908a8ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:28.769016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:28.770322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 21 15:40:28.771119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:28.771718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 217/436] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:28.778137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6ed35-4d1d-4c52-aff7-6851b908a8ff None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:28.823298 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a9700-b961-4b25-b464-c609c7cfa6bd None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:28.849035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:28.875580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec42bdd-b856-4ddd-b993-f6dc0ddb8773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:28.876412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec42bdd-b856-4ddd-b993-f6dc0ddb8773 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.876694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec42bdd-b856-4ddd-b993-f6dc0ddb8773 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.876969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec42bdd-b856-4ddd-b993-f6dc0ddb8773 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.892789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec42bdd-b856-4ddd-b993-f6dc0ddb8773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-7ec42bdd-b856-4ddd-b993-f6dc0ddb8773 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:28.902306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:28.903241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:28.904001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 220/437] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:28.909675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec42bdd-b856-4ddd-b993-f6dc0ddb8773 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:28.920605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6ed35-4d1d-4c52-aff7-6851b908a8ff None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:28.974224 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:28.975079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:28.975438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:28.975829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:28.976625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:28.977001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:28.978781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:28.992095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 218/438] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:28.998497 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.025656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.043201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec42bdd-b856-4ddd-b993-f6dc0ddb8773 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.078639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.079782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.080252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.080594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.081440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:29.081756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.083171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.091836 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 221/439] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:29.097397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.119461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff69c13-3e50-4181-aa15-179eda7349c5 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.132978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.157757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b11e1-cae7-412b-8bee-1a5209db14fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.158464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b11e1-cae7-412b-8bee-1a5209db14fc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.158773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b11e1-cae7-412b-8bee-1a5209db14fc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.159021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b11e1-cae7-412b-8bee-1a5209db14fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.214964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9dd87b-2dc2-477a-ad99-100c0f152b92 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.230833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b11e1-cae7-412b-8bee-1a5209db14fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.232291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b11e1-cae7-412b-8bee-1a5209db14fc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.258548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184711d3-4181-4e98-b381-18b2dba36fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.259291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184711d3-4181-4e98-b381-18b2dba36fd3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.259758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184711d3-4181-4e98-b381-18b2dba36fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.260044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184711d3-4181-4e98-b381-18b2dba36fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.278400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.335488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184711d3-4181-4e98-b381-18b2dba36fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.336982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184711d3-4181-4e98-b381-18b2dba36fd3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.349823 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 219/440] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:29.350746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf8a7-232d-4f19-a51c-d072eae9086a None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.351036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf8a7-232d-4f19-a51c-d072eae9086a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.351316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faf8a7-232d-4f19-a51c-d072eae9086a None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.352852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 220/441] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:29.354728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faf8a7-232d-4f19-a51c-d072eae9086a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.369274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.437255 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 222/442] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 15:40:29.438647 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07233a9a-4cb8-4750-8642-d79168468105 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.438932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07233a9a-4cb8-4750-8642-d79168468105 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.439204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07233a9a-4cb8-4750-8642-d79168468105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.455851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-07233a9a-4cb8-4750-8642-d79168468105 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:29.457056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-07233a9a-4cb8-4750-8642-d79168468105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:29.457744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faf8a7-232d-4f19-a51c-d072eae9086a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.492508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f38a6-5a0b-4d44-adba-fca6a0adc485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.493494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f38a6-5a0b-4d44-adba-fca6a0adc485 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.494034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f38a6-5a0b-4d44-adba-fca6a0adc485 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.494312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f38a6-5a0b-4d44-adba-fca6a0adc485 None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.495545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f38a6-5a0b-4d44-adba-fca6a0adc485 None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.497021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f38a6-5a0b-4d44-adba-fca6a0adc485 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.498756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 221/443] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:29.499578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f38a6-5a0b-4d44-adba-fca6a0adc485 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.501150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.527622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 223/444] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:29.528465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07233a9a-4cb8-4750-8642-d79168468105 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.582896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.601645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f38a6-5a0b-4d44-adba-fca6a0adc485 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.630869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07233a9a-4cb8-4750-8642-d79168468105 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.634267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e9968-c9b4-447d-a2f9-78e89c0d8cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.635308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e9968-c9b4-447d-a2f9-78e89c0d8cea None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.635616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e9968-c9b4-447d-a2f9-78e89c0d8cea None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.635926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e9968-c9b4-447d-a2f9-78e89c0d8cea None admin] PATH_INFO: `/v3/projects/20a1163a83aa4705a89b4635d4e037b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.637117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e9968-c9b4-447d-a2f9-78e89c0d8cea None admin] RBAC: Authorizing `identity:get_project(project_id=20a1163a83aa4705a89b4635d4e037b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.638602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e9968-c9b4-447d-a2f9-78e89c0d8cea None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.640307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 222/445] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/projects/20a1163a83aa4705a89b4635d4e037b4 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:29.641059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e9968-c9b4-447d-a2f9-78e89c0d8cea None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.670610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bade4-d38b-48ac-86d8-cd5636b7a8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.671472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bade4-d38b-48ac-86d8-cd5636b7a8b2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.671752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bade4-d38b-48ac-86d8-cd5636b7a8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.672021 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bade4-d38b-48ac-86d8-cd5636b7a8b2 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.673181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bade4-d38b-48ac-86d8-cd5636b7a8b2 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.674660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bade4-d38b-48ac-86d8-cd5636b7a8b2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.676278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 224/446] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:29.677018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bade4-d38b-48ac-86d8-cd5636b7a8b2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.721611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.746800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e9968-c9b4-447d-a2f9-78e89c0d8cea None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.776155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bade4-d38b-48ac-86d8-cd5636b7a8b2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.782026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9192-4b30-4e1a-935c-56428236ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.782826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9192-4b30-4e1a-935c-56428236ed9d None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.783109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9192-4b30-4e1a-935c-56428236ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.783388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c9192-4b30-4e1a-935c-56428236ed9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.793724 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c9192-4b30-4e1a-935c-56428236ed9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-2b8c9192-4b30-4e1a-935c-56428236ed9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:29.801368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:29.802258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:29.803009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 223/447] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:29.803641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9192-4b30-4e1a-935c-56428236ed9d None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.814846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456dd74-b770-4816-a570-11e2dc16c9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.815741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456dd74-b770-4816-a570-11e2dc16c9f6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.816017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456dd74-b770-4816-a570-11e2dc16c9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.816285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456dd74-b770-4816-a570-11e2dc16c9f6 None admin] PATH_INFO: `/v3/groups/3e6e4ab335da48048aced67d17c2c2ef` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.817479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456dd74-b770-4816-a570-11e2dc16c9f6 None admin] RBAC: Authorizing `identity:get_group(group_id=3e6e4ab335da48048aced67d17c2c2ef)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.817767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.818959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456dd74-b770-4816-a570-11e2dc16c9f6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.821342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 225/448] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/groups/3e6e4ab335da48048aced67d17c2c2ef => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:29.822305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456dd74-b770-4816-a570-11e2dc16c9f6 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.910726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c9192-4b30-4e1a-935c-56428236ed9d None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.922742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456dd74-b770-4816-a570-11e2dc16c9f6 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:29.937122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:29.945095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.945766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.946071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.946346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.947238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] RBAC: Adding query filter params (group.id=3e6e4ab335da48048aced67d17c2c2ef, role.id=ff19e0e651ce4a5a84fa63e1b3f386a5, scope.project.id=20a1163a83aa4705a89b4635d4e037b4) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:29.947614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.949017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.955086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 224/449] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/role_assignments?group.id=3e6e4ab335da48048aced67d17c2c2ef&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:29.955688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:29.974857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612da660-7eca-47fb-8932-6921beed276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:29.975548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612da660-7eca-47fb-8932-6921beed276e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:29.975826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612da660-7eca-47fb-8932-6921beed276e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:29.976094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612da660-7eca-47fb-8932-6921beed276e None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:29.976868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612da660-7eca-47fb-8932-6921beed276e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:29.977179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612da660-7eca-47fb-8932-6921beed276e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:29.978766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612da660-7eca-47fb-8932-6921beed276e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:29.988656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 226/450] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:29.994833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612da660-7eca-47fb-8932-6921beed276e None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:30.043115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:30.078767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558012c4-2a8c-4d49-b939-a5c408726ebe None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:30.111224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612da660-7eca-47fb-8932-6921beed276e None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:30.116512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:30.117187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:30.117465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:30.117734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:30.118729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] RBAC: Adding query filter params (group.id=3e6e4ab335da48048aced67d17c2c2ef, role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=20a1163a83aa4705a89b4635d4e037b4) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:30.119180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:30.120868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:30.127442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 225/451] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/role_assignments?group.id=3e6e4ab335da48048aced67d17c2c2ef&scope.project.id=20a1163a83aa4705a89b4635d4e037b4&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:30.153898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a99a-6261-4894-bc40-babd370e9a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:30.154591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a99a-6261-4894-bc40-babd370e9a22 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:30.156242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a99a-6261-4894-bc40-babd370e9a22 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:30.156242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934a99a-6261-4894-bc40-babd370e9a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:30.211861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:30.245616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934a99a-6261-4894-bc40-babd370e9a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:30.245616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934a99a-6261-4894-bc40-babd370e9a22 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:30.359141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 227/452] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:30.364389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:30.480203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8860748-d1f3-4660-9bf3-2ff0f1fda968 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:30.515645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c3d82-caae-4203-8620-e212375211d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:30.516545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c3d82-caae-4203-8620-e212375211d7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:30.516815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c3d82-caae-4203-8620-e212375211d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:30.517056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c3d82-caae-4203-8620-e212375211d7 None admin] PATH_INFO: `/v3/projects/eb46a2622cdd481281f00889cb2ab74f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:30.518277 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c3d82-caae-4203-8620-e212375211d7 None admin] RBAC: Authorizing `identity:get_project(project_id=eb46a2622cdd481281f00889cb2ab74f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:30.519736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c3d82-caae-4203-8620-e212375211d7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:30.521645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 226/453] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:40:30 2022] GET /identity/v3/projects/eb46a2622cdd481281f00889cb2ab74f => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:30.527523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a99a-6261-4894-bc40-babd370e9a22 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:30.601359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:30.628838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934a99a-6261-4894-bc40-babd370e9a22 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:30.666005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458209a4-8a5d-4d83-8350-bcd2ef8d17a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:30.666662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458209a4-8a5d-4d83-8350-bcd2ef8d17a7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:30.666943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458209a4-8a5d-4d83-8350-bcd2ef8d17a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:30.667223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458209a4-8a5d-4d83-8350-bcd2ef8d17a7 None admin] PATH_INFO: `/v3/groups/1a37c2d4d7da41808bc8e8f7d080e53c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:30.668524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458209a4-8a5d-4d83-8350-bcd2ef8d17a7 None admin] RBAC: Authorizing `identity:get_group(group_id=1a37c2d4d7da41808bc8e8f7d080e53c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:30.670024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458209a4-8a5d-4d83-8350-bcd2ef8d17a7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:30.671625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 228/454] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Jun 21 15:40:30 2022] GET /identity/v3/groups/1a37c2d4d7da41808bc8e8f7d080e53c => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:30.681469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c3d82-caae-4203-8620-e212375211d7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:30.685705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:30.797708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c3d82-caae-4203-8620-e212375211d7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:30.833397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8801a-6769-4234-9382-f7db5ef01459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:30.834087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8801a-6769-4234-9382-f7db5ef01459 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:30.834366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8801a-6769-4234-9382-f7db5ef01459 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:30.834641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8801a-6769-4234-9382-f7db5ef01459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:30.835477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8801a-6769-4234-9382-f7db5ef01459 None admin] RBAC: Adding query filter params (group.id=1a37c2d4d7da41808bc8e8f7d080e53c, role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=eb46a2622cdd481281f00889cb2ab74f) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:30.835788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8801a-6769-4234-9382-f7db5ef01459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:30.837250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8801a-6769-4234-9382-f7db5ef01459 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:30.843370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 227/455] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Jun 21 15:40:30 2022] GET /identity/v3/role_assignments?group.id=1a37c2d4d7da41808bc8e8f7d080e53c&scope.project.id=eb46a2622cdd481281f00889cb2ab74f&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:33.291760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7526e-7601-4c42-b113-04dfd54f2d03 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:33.292288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7526e-7601-4c42-b113-04dfd54f2d03 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:33.292288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7526e-7601-4c42-b113-04dfd54f2d03 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:33.294096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 229/456] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:33.299971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a9957-d3dc-4e3b-8b22-0a3c865c7a66 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:33.300806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a9957-d3dc-4e3b-8b22-0a3c865c7a66 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:33.301293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a9957-d3dc-4e3b-8b22-0a3c865c7a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:33.321876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6e9a9957-d3dc-4e3b-8b22-0a3c865c7a66 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:33.323417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6e9a9957-d3dc-4e3b-8b22-0a3c865c7a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:33.413388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:33.419424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 228/457] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:33.468679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7526e-7601-4c42-b113-04dfd54f2d03 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:33.592471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f945eae-60a0-4034-87cb-9b61d102966d None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:33.593116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f945eae-60a0-4034-87cb-9b61d102966d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:33.593587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f945eae-60a0-4034-87cb-9b61d102966d None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:33.595441 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 229/458] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:33.601066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e512e-5c9e-4bf5-bfa9-cbd8f3e7fb4b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:33.601659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e512e-5c9e-4bf5-bfa9-cbd8f3e7fb4b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:33.602190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e512e-5c9e-4bf5-bfa9-cbd8f3e7fb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:33.642961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2b6e512e-5c9e-4bf5-bfa9-cbd8f3e7fb4b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:33.646322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2b6e512e-5c9e-4bf5-bfa9-cbd8f3e7fb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:33.666699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7526e-7601-4c42-b113-04dfd54f2d03 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:33.742236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:33.743051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:33.743337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:33.743691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:33.765703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:33.767437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:33.769510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:33.780026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 230/459] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:33.781683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c218-5b67-4727-8e8a-70d84aa91a85 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:33.782136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c218-5b67-4727-8e8a-70d84aa91a85 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:33.782556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64c218-5b67-4727-8e8a-70d84aa91a85 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:33.784346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 231/460] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:33.798627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:33.799725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:33.800759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:33.800759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:33.800759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:33.800759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:33.804231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa556f-aa68-4adc-9bb9-2a6b837dca0e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:33.804522 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa556f-aa68-4adc-9bb9-2a6b837dca0e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:33.804801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa556f-aa68-4adc-9bb9-2a6b837dca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:33.806606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 230/461] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:33.807768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:33.828766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-02fa556f-aa68-4adc-9bb9-2a6b837dca0e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:33.830424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-02fa556f-aa68-4adc-9bb9-2a6b837dca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:33.913937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 232/462] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:33.915136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa556f-aa68-4adc-9bb9-2a6b837dca0e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:33.936465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc6c90-9925-465e-96ea-b48642779375 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:33.979920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:33.981181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:33.981685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:33.982190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:33.993147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:33.995014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.000176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:34.014744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:34.015595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:34.016410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:34.016410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:34.016410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:34.016410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:34.018358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 231/463] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:34.019357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:34.051849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa556f-aa68-4adc-9bb9-2a6b837dca0e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:34.072409 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:34.095100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:34.095780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.096059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.096327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.097051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:34.097351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:34.098787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.110708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 233/464] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:34.111506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:34.135768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cf708-2b10-4f03-99f0-ffa61b686693 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:34.175911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221c8d2-5b95-4d75-93c7-128af46da339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:34.177005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221c8d2-5b95-4d75-93c7-128af46da339 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.177465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221c8d2-5b95-4d75-93c7-128af46da339 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.177985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221c8d2-5b95-4d75-93c7-128af46da339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.189154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221c8d2-5b95-4d75-93c7-128af46da339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:34.191045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221c8d2-5b95-4d75-93c7-128af46da339 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-5221c8d2-5b95-4d75-93c7-128af46da339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:34.210470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:34.211371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:34.212514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:34.212514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:34.212514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:34.212514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:34.212514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 232/465] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:34.213214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94468620-8ed1-4616-b077-d59b7b18150c None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.213440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94468620-8ed1-4616-b077-d59b7b18150c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.213726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94468620-8ed1-4616-b077-d59b7b18150c None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.219287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 233/466] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:34.219287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94468620-8ed1-4616-b077-d59b7b18150c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:34.220044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac84594-df4c-482b-87af-b5b620133fb9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:34.238300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:34.260382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:34.261073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.261358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.261643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.262403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:34.262715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:34.264135 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.281425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 234/467] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:34.281425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:34.337413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:34.344367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94468620-8ed1-4616-b077-d59b7b18150c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:34.381044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926c317-6357-45bf-9aea-27132f87f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:34.382128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926c317-6357-45bf-9aea-27132f87f446 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.382413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926c317-6357-45bf-9aea-27132f87f446 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.382689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926c317-6357-45bf-9aea-27132f87f446 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.383123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef31262-ba11-474b-89a2-caa33d74db96 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:34.383566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926c317-6357-45bf-9aea-27132f87f446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:34.385180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926c317-6357-45bf-9aea-27132f87f446 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.408105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 234/468] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:40:34 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:34.419490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae14a-60aa-4234-94e6-3b12eef2dcac None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.419810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae14a-60aa-4234-94e6-3b12eef2dcac None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.420303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ae14a-60aa-4234-94e6-3b12eef2dcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.425050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:34.425748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.426119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.426335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.427081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:34.427388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:34.428853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.451134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 235/469] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:34.452011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:34.453859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-556ae14a-60aa-4234-94e6-3b12eef2dcac None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:34.455116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-556ae14a-60aa-4234-94e6-3b12eef2dcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:34.503592 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:34.549360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 235/470] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:34.550159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ae14a-60aa-4234-94e6-3b12eef2dcac None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:34.597254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c990e5-7ce3-4da3-a62f-577887cd8978 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:34.645381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00e598-9cb6-45ce-adeb-dea32bc6bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:34.647956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00e598-9cb6-45ce-adeb-dea32bc6bf05 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.647956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00e598-9cb6-45ce-adeb-dea32bc6bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.647956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00e598-9cb6-45ce-adeb-dea32bc6bf05 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.647956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00e598-9cb6-45ce-adeb-dea32bc6bf05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:34.649241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00e598-9cb6-45ce-adeb-dea32bc6bf05 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.671826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:34.676346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 236/471] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:40:34 2022] POST /identity/v3/users => generated 248 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:34.677129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00e598-9cb6-45ce-adeb-dea32bc6bf05 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:34.704541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ae14a-60aa-4234-94e6-3b12eef2dcac None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:34.749979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231b2e6-3073-4723-8ee5-4e49f147f82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:34.750690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231b2e6-3073-4723-8ee5-4e49f147f82f None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.750970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231b2e6-3073-4723-8ee5-4e49f147f82f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.751247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231b2e6-3073-4723-8ee5-4e49f147f82f None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.752424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231b2e6-3073-4723-8ee5-4e49f147f82f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:34.753592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231b2e6-3073-4723-8ee5-4e49f147f82f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.786087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 236/472] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:40:34 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:34.788225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593315f-ec84-46bb-a4e0-987af6862ef2 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.788721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593315f-ec84-46bb-a4e0-987af6862ef2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.789325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593315f-ec84-46bb-a4e0-987af6862ef2 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.791588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 237/473] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:34.797697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc81ebb-3dd1-4058-9656-4549a632bbb4 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.798333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc81ebb-3dd1-4058-9656-4549a632bbb4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.798845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc81ebb-3dd1-4058-9656-4549a632bbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.826045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:34.828736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-bdc81ebb-3dd1-4058-9656-4549a632bbb4 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:34.830308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bdc81ebb-3dd1-4058-9656-4549a632bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:34.832294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00e598-9cb6-45ce-adeb-dea32bc6bf05 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:34.874905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:34.875722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:34.876035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:34.876274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:34.896245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:34.898802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:34.920672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:34.921763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:34.922787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:34.922787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:34.922787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:34.922787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:34.924771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 237/474] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:34.932870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:34.977009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 238/475] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:35.003936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc81ebb-3dd1-4058-9656-4549a632bbb4 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:35.093683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce9def-af26-4790-acf3-ef510c3a05cf None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:35.105103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:35.131281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:35.132033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:35.132407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:35.132700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:35.133448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:35.133759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:35.135515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:35.142097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc81ebb-3dd1-4058-9656-4549a632bbb4 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:35.150179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 238/476] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:35.155830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:35.165825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:35.184842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:35.185657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:35.186180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:35.186518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:35.187344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:35.189071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:35.199778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.sql.core [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 15:40:35.199778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 15:40:35.199778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [parameters: {'id': 'bc225cceb18648f78740bb808f628721', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 15:40:35.199778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86657) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application conn.query(q) Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application result.read() Jun 21 15:40:35.219111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return ref.to_dict() Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 15:40:35.220349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application next(self.gen) Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application yield resource Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application next(self.gen) Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise self.value Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application session.commit() Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 15:40:35.221573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self.session.flush() Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application compat.raise_( Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 15:40:35.222976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 15:40:35.224528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application conn.query(q) Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application result.read() Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 15:40:35.226231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [parameters: {'id': 'bc225cceb18648f78740bb808f628721', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:35.227955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 21 15:40:35.229738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:35.232424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 239/477] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jun 21 15:40:35 2022] POST /identity/v3/roles => generated 144 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:35.232424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:35.312526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1eab7-0271-4313-8af2-d1d2325a58fc None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:35.380992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d4ab3-c1b3-4e9b-9ad1-a6e0ac6ba039 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:35.391652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da425a4-e77f-4913-96bd-96852fea85e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:35.392358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da425a4-e77f-4913-96bd-96852fea85e0 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:35.392639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da425a4-e77f-4913-96bd-96852fea85e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:35.392917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da425a4-e77f-4913-96bd-96852fea85e0 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:35.393716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da425a4-e77f-4913-96bd-96852fea85e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:35.395268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da425a4-e77f-4913-96bd-96852fea85e0 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:35.434362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe5391-6f75-428f-be6a-7d5afed6d6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:35.435433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe5391-6f75-428f-be6a-7d5afed6d6bc None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:35.435713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe5391-6f75-428f-be6a-7d5afed6d6bc None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:35.435999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe5391-6f75-428f-be6a-7d5afed6d6bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:35.438406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:35.441561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 239/478] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:40:35 2022] POST /identity/v3/users => generated 249 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:35.449560 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe5391-6f75-428f-be6a-7d5afed6d6bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-32fe5391-6f75-428f-be6a-7d5afed6d6bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:35.458672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:35.459656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:35.460469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 240/479] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:40:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:35.466669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da425a4-e77f-4913-96bd-96852fea85e0 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:35.578768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:35.637305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da425a4-e77f-4913-96bd-96852fea85e0 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:35.673055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c4d49-f05d-41f9-9301-6550a1da26e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:35.673738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4d49-f05d-41f9-9301-6550a1da26e4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:35.674048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4d49-f05d-41f9-9301-6550a1da26e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:35.674323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4d49-f05d-41f9-9301-6550a1da26e4 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:35.675093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c4d49-f05d-41f9-9301-6550a1da26e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:35.675400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c4d49-f05d-41f9-9301-6550a1da26e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:35.676863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c4d49-f05d-41f9-9301-6550a1da26e4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:35.686215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 240/480] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:37.207521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040a07-d0ee-4fce-9dbc-85a92f8f5d67 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.208546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040a07-d0ee-4fce-9dbc-85a92f8f5d67 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.209089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c040a07-d0ee-4fce-9dbc-85a92f8f5d67 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.210991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 241/481] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:37.216831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779883-ea2a-4002-b733-f9880fc46ae9 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.217895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779883-ea2a-4002-b733-f9880fc46ae9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.218376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779883-ea2a-4002-b733-f9880fc46ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.247770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-9e779883-ea2a-4002-b733-f9880fc46ae9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:37.249434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9e779883-ea2a-4002-b733-f9880fc46ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:37.261053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2b30a-0701-4594-9bc7-737863c1be70 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.261334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2b30a-0701-4594-9bc7-737863c1be70 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.261605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2b30a-0701-4594-9bc7-737863c1be70 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.264253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 242/482] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:37.268344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b118840-1063-44ab-b055-05592ae7f700 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.268633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b118840-1063-44ab-b055-05592ae7f700 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.268913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b118840-1063-44ab-b055-05592ae7f700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.289275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7b118840-1063-44ab-b055-05592ae7f700 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:37.290546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7b118840-1063-44ab-b055-05592ae7f700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:37.352832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 241/483] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:37.376699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e779883-ea2a-4002-b733-f9880fc46ae9 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:37.440370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:37.463565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 243/484] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:37.487342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b118840-1063-44ab-b055-05592ae7f700 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:37.517798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e779883-ea2a-4002-b733-f9880fc46ae9 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:37.587676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5827c-1548-4e90-b389-2d3e0a042f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:37.588498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5827c-1548-4e90-b389-2d3e0a042f3a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.588779 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5827c-1548-4e90-b389-2d3e0a042f3a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.589053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5827c-1548-4e90-b389-2d3e0a042f3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.604641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5827c-1548-4e90-b389-2d3e0a042f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-3ff5827c-1548-4e90-b389-2d3e0a042f3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:37.612524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:37.613619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:37.614636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 242/485] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:37.615313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7d90-61be-46de-ae12-053dff4d8356 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.615769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7d90-61be-46de-ae12-053dff4d8356 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.615864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7d90-61be-46de-ae12-053dff4d8356 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.622231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 243/486] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:37.623059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d7d90-61be-46de-ae12-053dff4d8356 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:37.625970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b118840-1063-44ab-b055-05592ae7f700 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:37.653249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:37.666656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4257b6-ec4b-4359-be89-4d08e54f34b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:37.667726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4257b6-ec4b-4359-be89-4d08e54f34b1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.668207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4257b6-ec4b-4359-be89-4d08e54f34b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.668672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4257b6-ec4b-4359-be89-4d08e54f34b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.680429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4257b6-ec4b-4359-be89-4d08e54f34b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-ec4257b6-ec4b-4359-be89-4d08e54f34b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:37.689448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:37.690570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:37.691544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:37.693955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 244/487] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:37.695377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a3bfb-657c-4968-a11e-62a5ed77e45c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.695804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a3bfb-657c-4968-a11e-62a5ed77e45c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.696296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a3bfb-657c-4968-a11e-62a5ed77e45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.714962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-290a3bfb-657c-4968-a11e-62a5ed77e45c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:37.716504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-290a3bfb-657c-4968-a11e-62a5ed77e45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:37.729655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d7d90-61be-46de-ae12-053dff4d8356 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:37.744307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:37.765118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:37.765842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.766176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.766452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.767203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:37.767508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:37.768971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:37.778369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 244/488] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:37.783627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:37.796590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 245/489] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:37.797821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a3bfb-657c-4968-a11e-62a5ed77e45c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:37.888859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428d445-906d-487c-9065-27a8c1521921 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:37.920375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a3bfb-657c-4968-a11e-62a5ed77e45c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:37.926987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:37.927674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.927992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.928263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.928991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:37.929303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:37.930928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:37.941023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 245/490] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:37.941993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:37.943072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:37.968425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:37.969510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:37.969964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:37.970379 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:37.982246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:37.983894 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:37.985103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:38.006824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:38.007933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:38.008981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:38.008981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:38.008981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:38.008981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:38.010953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 246/491] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:38.011930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.054762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb211-2db8-4b7b-b9af-c5e545ae4a01 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.091812 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aa74d-534b-405a-8d27-a4ca59e6a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.092629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa74d-534b-405a-8d27-a4ca59e6a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.092913 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa74d-534b-405a-8d27-a4ca59e6a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.093186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa74d-534b-405a-8d27-a4ca59e6a4f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.105508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2aa74d-534b-405a-8d27-a4ca59e6a4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-4d2aa74d-534b-405a-8d27-a4ca59e6a4f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:38.113265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:38.114315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 246/492] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:38.120133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aa74d-534b-405a-8d27-a4ca59e6a4f4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.128892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc773b-9985-4a34-93ca-7676ee208f69 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.166093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a223ff-dd58-400a-a3f3-9495886f7982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.167008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a223ff-dd58-400a-a3f3-9495886f7982 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.167377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a223ff-dd58-400a-a3f3-9495886f7982 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.167724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a223ff-dd58-400a-a3f3-9495886f7982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.178630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a223ff-dd58-400a-a3f3-9495886f7982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:38.180182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a223ff-dd58-400a-a3f3-9495886f7982 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-69a223ff-dd58-400a-a3f3-9495886f7982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:38.198897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:38.199875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:38.200822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:38.200822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:38.200822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:38.200822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:38.202603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 247/493] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:38.204094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613f2be-b826-48fe-b28f-0a45743e8822 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.204503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613f2be-b826-48fe-b28f-0a45743e8822 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.204906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613f2be-b826-48fe-b28f-0a45743e8822 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.206352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 248/494] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:38.207532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613f2be-b826-48fe-b28f-0a45743e8822 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.229066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:38.231261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:38.242752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aa74d-534b-405a-8d27-a4ca59e6a4f4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.279309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.279983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.280265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.280544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.281268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:38.281571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:38.282994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.295493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 247/495] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:38.296329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.320669 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613f2be-b826-48fe-b28f-0a45743e8822 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.358144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e537c-1d6c-4520-b829-f0d233d5c372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.358810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e537c-1d6c-4520-b829-f0d233d5c372 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.359093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e537c-1d6c-4520-b829-f0d233d5c372 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.359367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e537c-1d6c-4520-b829-f0d233d5c372 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.360130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e537c-1d6c-4520-b829-f0d233d5c372 None admin] RBAC: Adding query filter params (name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:38.360436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e537c-1d6c-4520-b829-f0d233d5c372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:38.361851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e537c-1d6c-4520-b829-f0d233d5c372 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.369864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 249/496] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:38.371065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e651cf9-2780-4654-a02f-590873c5194f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.371412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e651cf9-2780-4654-a02f-590873c5194f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.371626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e651cf9-2780-4654-a02f-590873c5194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.390406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7e651cf9-2780-4654-a02f-590873c5194f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:38.391612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7e651cf9-2780-4654-a02f-590873c5194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:38.398754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c3ac-56c2-4aa3-8bf3-9e918994ae3e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.433974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc55eb-00b0-4b4f-8d58-b696cefddc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.434642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc55eb-00b0-4b4f-8d58-b696cefddc04 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.434924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc55eb-00b0-4b4f-8d58-b696cefddc04 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.435197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc55eb-00b0-4b4f-8d58-b696cefddc04 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.435922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc55eb-00b0-4b4f-8d58-b696cefddc04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:38.436224 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc55eb-00b0-4b4f-8d58-b696cefddc04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:38.437619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc55eb-00b0-4b4f-8d58-b696cefddc04 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.449374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 248/497] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:38.450675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5a59-dd33-4249-be55-09ce031bdb07 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.450965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5a59-dd33-4249-be55-09ce031bdb07 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.451242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5a59-dd33-4249-be55-09ce031bdb07 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.452475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 249/498] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:38.453155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5a59-dd33-4249-be55-09ce031bdb07 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.465261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 250/499] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:38.466140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e651cf9-2780-4654-a02f-590873c5194f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.473182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:38.483369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:38.561423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5a59-dd33-4249-be55-09ce031bdb07 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.573721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e651cf9-2780-4654-a02f-590873c5194f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.596048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.596859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.597142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.597418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.604960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:38.606467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.607266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bb422-7cab-4a13-be85-21ca30d4980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.608055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bb422-7cab-4a13-be85-21ca30d4980b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.608342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bb422-7cab-4a13-be85-21ca30d4980b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.608614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bb422-7cab-4a13-be85-21ca30d4980b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.619102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bb422-7cab-4a13-be85-21ca30d4980b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:38.620541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bb422-7cab-4a13-be85-21ca30d4980b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:38.622019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:38.622865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 15:40:38.623808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:38.627592 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 250/500] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:38.628630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-9d5bb422-7cab-4a13-be85-21ca30d4980b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:38.638892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:38.639764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:38.640582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:38.640582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:38.640582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:38.640582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:38.640582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 251/501] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:38.642952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4cfd-3b1b-4f85-9fcb-1cc676b91707 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.643253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4cfd-3b1b-4f85-9fcb-1cc676b91707 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.643526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4cfd-3b1b-4f85-9fcb-1cc676b91707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.660297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-651e4cfd-3b1b-4f85-9fcb-1cc676b91707 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:38.661479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-651e4cfd-3b1b-4f85-9fcb-1cc676b91707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:38.712021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:38.719891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:38.731803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 252/502] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:38.732636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4cfd-3b1b-4f85-9fcb-1cc676b91707 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.738206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e05c3-5890-4720-9619-92a9ec33fed6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.775739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.776547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.776855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.777205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.786035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:38.787541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:38.802137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:38.803236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:38.809261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 251/503] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:38.813627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.854377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4cfd-3b1b-4f85-9fcb-1cc676b91707 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.893095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20581149-c31e-47a7-8870-99d02bff282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.893898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20581149-c31e-47a7-8870-99d02bff282c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.894176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20581149-c31e-47a7-8870-99d02bff282c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.894447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20581149-c31e-47a7-8870-99d02bff282c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.906832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20581149-c31e-47a7-8870-99d02bff282c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-20581149-c31e-47a7-8870-99d02bff282c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:38.914818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:38.915866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 253/504] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:38.921942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20581149-c31e-47a7-8870-99d02bff282c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:38.934426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea87a74-b537-451f-922d-d09d45458be7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:38.968539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:38.969204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:38.969484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:38.969759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:38.970631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:38.970937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:38.971942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:38.972396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:38.990137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.011052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 252/505] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:39.012005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.027729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20581149-c31e-47a7-8870-99d02bff282c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.064887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.065558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.065873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.066152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.066876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:39.067183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.068580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.088117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 254/506] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:39.088936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.122142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a60cb-56dd-4b08-afe5-32d3388a5198 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.160846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.161621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.161972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.162249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.172460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.174025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:39.192024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:39.193023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:39.194147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:39.194147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:39.194147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:39.194147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:39.194147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 253/507] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:39.194847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.204657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a87f79-a46f-4937-8a58-f2c86a8e30fd None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.218689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.241948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.242996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.243317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.243592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.244670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:39.244980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.246709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.251069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.285451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 255/508] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:39.286325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.309986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcd3ae-ccf5-442f-9685-376047624f5c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.345230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.345930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.346211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.346479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.347226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:39.347549 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.348936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.357910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 254/509] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:39.358710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.386247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb59912-aa5d-43f3-9637-428b692ca51b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.422358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b38b0-92b8-46a8-b25f-521d553e0284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.423018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b38b0-92b8-46a8-b25f-521d553e0284 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.423307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b38b0-92b8-46a8-b25f-521d553e0284 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.423583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b38b0-92b8-46a8-b25f-521d553e0284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.458796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0f05e-cfad-4a55-b262-25ab6d00b3a1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.463167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.469449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.492910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b38b0-92b8-46a8-b25f-521d553e0284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.494613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b38b0-92b8-46a8-b25f-521d553e0284 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.495602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.496623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.496906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.498001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] PATH_INFO: `/v3/users/glance` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.507504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.509129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:39.525080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:39.526198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:39.527148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 255/510] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:39.527698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.614886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 256/511] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:39.615584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b38b0-92b8-46a8-b25f-521d553e0284 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.642244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a31946-8a70-472c-b58f-ea6b1ddcf91b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.676421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.677095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.677369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.677648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.678389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:39.678690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.680081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.687360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.692516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 256/512] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:39.693537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.720219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b38b0-92b8-46a8-b25f-521d553e0284 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.727505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.758079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c1d6a-2cb3-4088-a914-dfe4da3fcf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.758704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c1d6a-2cb3-4088-a914-dfe4da3fcf9d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.759023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c1d6a-2cb3-4088-a914-dfe4da3fcf9d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.759303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c1d6a-2cb3-4088-a914-dfe4da3fcf9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.799531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2b234-77bd-4783-9913-4eba45afedb8 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:39.829886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c1d6a-2cb3-4088-a914-dfe4da3fcf9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.831389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c1d6a-2cb3-4088-a914-dfe4da3fcf9d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.833285 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:39.834118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:39.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:39.834671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:39.845192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:39.846696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:39.864093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:39.865104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:39.866175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:39.866175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:39.866175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:39.866175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:39.866175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 257/513] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:39.866175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.902791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.936386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 257/514] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:39.937249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c1d6a-2cb3-4088-a914-dfe4da3fcf9d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:39.952813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:39.971777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f97432-aff7-4299-ba70-467a16857f41 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.005275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.006046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.006323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.006593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.007440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:40.007747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.009568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.027261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 258/515] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:40.028062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.044160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c1d6a-2cb3-4088-a914-dfe4da3fcf9d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.077531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.078370 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.078646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.078920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.089697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.091216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:40.110599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:40.111651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:40.112631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:40.112631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:40.112631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:40.112631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:40.116355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 258/516] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:40.117110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.132866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ecc85-075e-4864-8f70-a367ba8002d1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.142293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:40.168149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62228c65-794f-4a11-9c27-634a7d9e4d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.168705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62228c65-794f-4a11-9c27-634a7d9e4d48 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.168989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62228c65-794f-4a11-9c27-634a7d9e4d48 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.169268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62228c65-794f-4a11-9c27-634a7d9e4d48 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.170017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62228c65-794f-4a11-9c27-634a7d9e4d48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.171493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62228c65-794f-4a11-9c27-634a7d9e4d48 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.181518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:40.205802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 259/517] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:40:40 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:40:40.205802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62228c65-794f-4a11-9c27-634a7d9e4d48 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.253093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa12cf-de5d-438d-9d87-6eea692a6009 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.296064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.296623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.297052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.297338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.298080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:40.298386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.299832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.317476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 259/518] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:40.317476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.326269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62228c65-794f-4a11-9c27-634a7d9e4d48 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.365077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.365919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.366194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.366463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.379342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.380827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:40.402684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:40.403762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:40.404848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:40.404848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:40.404848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:40.404848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:40.404848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 260/519] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:40.404848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.416779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:40.439047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd849f2-b5b3-461e-aee3-2f027897758b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.460685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:40.476635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1733c-ef37-4268-babb-fd620e40f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.477967 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1733c-ef37-4268-babb-fd620e40f7ff None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.478520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1733c-ef37-4268-babb-fd620e40f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.478714 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1733c-ef37-4268-babb-fd620e40f7ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.520783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7557c-f24b-4755-88ec-ecdd5fa9de87 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.559577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.560172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.560459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.560668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd1733c-ef37-4268-babb-fd620e40f7ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.560770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.561451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:40.561762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.562200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd1733c-ef37-4268-babb-fd620e40f7ff None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.563181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.580250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 261/520] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:40.580250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.664350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:40.689998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 260/521] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:40.691019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1733c-ef37-4268-babb-fd620e40f7ff None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.705742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd1771-19f2-4895-8858-561e2022767c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.717304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:40.748421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.749262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.749542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.749844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.760126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.762129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:40.777834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:40.778874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 15:40:40.779888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:40.783686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 262/522] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:40.784583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.803822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1733c-ef37-4268-babb-fd620e40f7ff None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.839586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.840256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.840540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.840821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.841549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:40.841885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.843267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.855619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 261/523] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:40.856469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.897406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657b8f8-cf42-4437-95bd-dc857201335e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.907146 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:40.937305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:40.938194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:40.938500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:40.938779 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:40.946660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:40.948276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:40.964190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:40.965121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:40.965995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 263/524] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:40.966455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:40.973929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a8b82-6eb5-4992-a957-8c5f34144fa1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:40.996402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:41.011449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.012639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.012904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.015248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.027792 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.029270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:41.048898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:41.050105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:41.051444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:41.051444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:41.051444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:41.051444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:41.051444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 262/525] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:41.051444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.077290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e2d89-36f3-48d9-b41e-410ec01c8317 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.116186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.116864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.117160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.117420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.118354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:41.118696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.120418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.142876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 264/526] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Jun 21 15:40:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:41.142876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.157428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:41.165228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a186856-ed92-42f8-9fe6-441bb98a424c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.200537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.201599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.201917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.202257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.209239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.210742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:41.225810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:41.226768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:41.227665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 263/527] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:41.228753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.246714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:41.268542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ed0c5-f550-4bb8-bee0-8de64454f677 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.304762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.305752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.306064 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.306347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.307151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:41.307463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.308934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.327099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 265/528] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:41.327975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.346110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b68b-b4a2-4402-bc0a-27ffb8562786 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.386086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.386604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.386907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.387182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.387917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:41.388230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.389926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.403378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 264/529] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:41.404235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.421964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:41.442219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3122a-fd85-4ded-a44f-ac38f7f7e6ce None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.481414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f1d48-0324-4275-904a-72d69858590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.482120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f1d48-0324-4275-904a-72d69858590a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.482466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f1d48-0324-4275-904a-72d69858590a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.482748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f1d48-0324-4275-904a-72d69858590a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.499350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:41.511939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987ae79-e666-4057-a479-a0324fb2b0e1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.548539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.549261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.549545 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.549853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.550655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:41.550965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.552436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.559926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f1d48-0324-4275-904a-72d69858590a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.561401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f1d48-0324-4275-904a-72d69858590a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.571459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 265/530] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:41.574971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.666107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 266/531] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:41.666732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f1d48-0324-4275-904a-72d69858590a None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.675933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:41.703405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea996d-ca94-4474-a83f-4346b3a31932 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.718328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:41.740996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.741664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.741995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.742276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.743352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=ed0934c0b1a64dfca3431dc32441e2db) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:41.743730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.745266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.754110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 266/532] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/role_assignments?user.id=ed0934c0b1a64dfca3431dc32441e2db&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:41.756081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.778840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f1d48-0324-4275-904a-72d69858590a None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.825827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.826541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.826822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.827098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.835414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.837018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:41.852351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:41.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:41.854193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:41.854193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:41.854193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:41.854193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:41.854193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:41.854193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:41.854193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:41.854193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:41.854597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 267/533] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:41.855326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.921537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f058c87-05c8-4ff7-b2a1-5c6c7681a4c7 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.960894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:41.962066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:41.962530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:41.963058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:41.964055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=46e5ca80b7834ab7bc951b454c9aae49) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:41.964534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:41.966416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:41.978182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 267/534] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/role_assignments?user.id=46e5ca80b7834ab7bc951b454c9aae49&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:41.979198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:41.983782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96bf3a-0d52-4a7c-86f9-35a5b77faffa None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:41.988969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:41.994996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:42.030491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:42.031743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.032350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.032828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.049215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:42.051170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:42.072578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:42.073508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:42.074361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:42.074361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:42.074361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:42.074361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:42.077468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 268/535] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:42.087028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:42.167604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef161-1df1-46f2-9c08-5102e2e1ee65 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:42.235022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:42.236122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.236691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.239563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.240618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:42.241141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:42.242923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:42.273451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 268/536] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:42.279812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:42.284988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a57d2-2e0a-45c6-a199-266517df0110 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:42.329410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:42.331302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.331788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.332410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.333492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:42.334172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:42.336345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:42.351891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 269/537] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:42.358935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:42.386070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:42.394184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:42.435433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0b81-3c5e-4f4f-96a3-39c5f5781f9f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:42.478009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f6525-c4ad-4b2d-ad11-cc006e0a062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:42.479270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6525-c4ad-4b2d-ad11-cc006e0a062b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.479653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6525-c4ad-4b2d-ad11-cc006e0a062b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.479944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6525-c4ad-4b2d-ad11-cc006e0a062b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.480799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f6525-c4ad-4b2d-ad11-cc006e0a062b None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=5cb5dc4c5a5a448294c57a2db5938d9d) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:42.481112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f6525-c4ad-4b2d-ad11-cc006e0a062b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:42.482635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f6525-c4ad-4b2d-ad11-cc006e0a062b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:42.490015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83844935-1088-4c51-8fd2-2f56f775bfc8 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:42.494451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 269/538] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:42 2022] GET /identity/v3/role_assignments?user.id=5cb5dc4c5a5a448294c57a2db5938d9d&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:42.495984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f542635-1d80-462a-b565-fe4b0a679c2d None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.496278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f542635-1d80-462a-b565-fe4b0a679c2d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.496558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f542635-1d80-462a-b565-fe4b0a679c2d None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.502728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 270/539] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:42.516487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43959a-fee1-4552-872f-d6292f3d3555 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.516796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43959a-fee1-4552-872f-d6292f3d3555 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.517054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43959a-fee1-4552-872f-d6292f3d3555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.532804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:42.534995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.535534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.536051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.545016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:42.546957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:42.562845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-de43959a-fee1-4552-872f-d6292f3d3555 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:42.563259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:42.564098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:42.564902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:42.564902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:42.564902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:42.564902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:42.564902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:42.564902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:42.564902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:42.564902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:42.565295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-de43959a-fee1-4552-872f-d6292f3d3555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:42.567736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 270/540] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:42.573760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:42.691502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff6136-805a-41fe-9b38-06b4b5a3b6bc None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:42.732198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:42.737195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.737195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.737195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.737195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:42.737195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:42.742073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:42.743467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:42.761287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 271/541] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:42.770063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:42.783104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 271/542] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:42.822228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43959a-fee1-4552-872f-d6292f3d3555 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:42.926194 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:42.928526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74360dae-4f98-4669-8ba4-7125965c9167 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:42.975779 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:42.976745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:42.977168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:42.977643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:42.978718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=6efa4bed025741d492158b5c6e45c7e7) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:42.979209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:42.980878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:42.995652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 272/543] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:42 2022] GET /identity/v3/role_assignments?user.id=6efa4bed025741d492158b5c6e45c7e7&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:43.017756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43959a-fee1-4552-872f-d6292f3d3555 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:43.076940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215fb4c-513e-4efa-a5ac-7a49fb98f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:43.078074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215fb4c-513e-4efa-a5ac-7a49fb98f9aa None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:43.078494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215fb4c-513e-4efa-a5ac-7a49fb98f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:43.078928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215fb4c-513e-4efa-a5ac-7a49fb98f9aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:43.102871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215fb4c-513e-4efa-a5ac-7a49fb98f9aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-b215fb4c-513e-4efa-a5ac-7a49fb98f9aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:43.111132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:43.112165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:43.113135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:43.115482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 272/544] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:43.121845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:43.125016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:43.283620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e209b4b-cbf5-42c2-9bb5-61e6b5512163 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:43.333198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:43.334014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:43.334306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:43.334585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:43.335411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:43.335725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:43.337203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:43.346469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 273/545] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:43.352649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215fb4c-513e-4efa-a5ac-7a49fb98f9aa None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:43.499195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215fb4c-513e-4efa-a5ac-7a49fb98f9aa None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:43.519628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:43.541630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:43.542466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:43.542748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:43.543019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:43.562143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:43.563632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:43.584843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:43.585758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:43.587261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:43.587261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:43.587261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:43.587261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:43.587261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 273/546] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:43.592414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:43.685119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:43.738484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876abe-a496-4ab8-be2f-849e052ee3f6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:43.786726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79930e-e416-4265-8036-d549c265559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:43.787330 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79930e-e416-4265-8036-d549c265559f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:43.787615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79930e-e416-4265-8036-d549c265559f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:43.787897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79930e-e416-4265-8036-d549c265559f None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:43.788641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79930e-e416-4265-8036-d549c265559f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:43.788955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79930e-e416-4265-8036-d549c265559f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:43.790704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79930e-e416-4265-8036-d549c265559f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:43.805144 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 274/547] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:43.810746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:43.957827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc74d8-2675-407f-aa6b-8bfeea954822 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:44.010556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:44.011401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.011687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.011969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.020495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:44.022443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:44.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:44.043337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:44.044892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 274/548] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:44.045602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ecff-e3bf-481b-a50a-324050b9bacb None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.045602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ecff-e3bf-481b-a50a-324050b9bacb None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.045602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ecff-e3bf-481b-a50a-324050b9bacb None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.047110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 275/549] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:44.052617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:44.057898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366e3f-a866-453c-afb1-bd8a4471a0d8 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.058199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366e3f-a866-453c-afb1-bd8a4471a0d8 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.058501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366e3f-a866-453c-afb1-bd8a4471a0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.075401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-53366e3f-a866-453c-afb1-bd8a4471a0d8 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:44.076603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-53366e3f-a866-453c-afb1-bd8a4471a0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:44.106151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:44.182637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 276/550] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:44.214073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10808-c6a0-4ae7-9a34-b0ae8cbbcbc4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:44.221723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53366e3f-a866-453c-afb1-bd8a4471a0d8 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:44.257877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:44.260331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.260901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.263318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.264447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:44.264957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:44.268156 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:44.307921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 275/551] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jun 21 15:40:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:44.314486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:44.332635 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:44.396031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53366e3f-a866-453c-afb1-bd8a4471a0d8 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:44.417734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:44.434558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d6a2f-edbb-4931-84a4-85fb42889de6 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:44.441253 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e277a69-0d61-4f47-955f-1fd8aecb4400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:44.442277 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277a69-0d61-4f47-955f-1fd8aecb4400 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.442699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277a69-0d61-4f47-955f-1fd8aecb4400 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.443518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277a69-0d61-4f47-955f-1fd8aecb4400 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.456913 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e277a69-0d61-4f47-955f-1fd8aecb4400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-5e277a69-0d61-4f47-955f-1fd8aecb4400 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:44.468036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:44.469102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:44.470229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:44.472644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 277/552] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:44.474128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e682ee8-d28a-4754-914b-7e742a28f891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:44.474370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49ef0-b817-4f2b-8f9a-fd175a0b61a4 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.474791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49ef0-b817-4f2b-8f9a-fd175a0b61a4 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.475072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e682ee8-d28a-4754-914b-7e742a28f891 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.475355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49ef0-b817-4f2b-8f9a-fd175a0b61a4 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.475514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e682ee8-d28a-4754-914b-7e742a28f891 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.475879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e682ee8-d28a-4754-914b-7e742a28f891 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.477706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 278/553] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:44.478733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49ef0-b817-4f2b-8f9a-fd175a0b61a4 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:44.553282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e682ee8-d28a-4754-914b-7e742a28f891 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:44.554815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e682ee8-d28a-4754-914b-7e742a28f891 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:44.621287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49ef0-b817-4f2b-8f9a-fd175a0b61a4 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:44.638901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:44.652247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:44.663127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:44.663999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.664386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.664884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.666033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:44.666451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:44.668077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:44.673315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 276/554] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 21 15:40:44.674764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4080c-2678-4395-bf51-4f6b7df137c5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.675057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4080c-2678-4395-bf51-4f6b7df137c5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.675338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4080c-2678-4395-bf51-4f6b7df137c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.687317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 279/555] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:44.687317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:44.698317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8db4080c-2678-4395-bf51-4f6b7df137c5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:44.699614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8db4080c-2678-4395-bf51-4f6b7df137c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:44.774400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 277/556] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:44.782076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4080c-2678-4395-bf51-4f6b7df137c5 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:44.825879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5ab7f-9eec-48a2-a69f-fbfa374d46f2 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:44.876238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:44.877949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.878264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.878542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.897534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:44.900878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:44.916867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4080c-2678-4395-bf51-4f6b7df137c5 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:44.921501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:44.922393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:44.923227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:44.923227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:44.923227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:44.923227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:44.929820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 280/557] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:44.931009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:44.937208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:44.962851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:44.963712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:44.964683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:44.964809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:44.966462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:44.984209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:44.985924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:45.011002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:45.012456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:45.013305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:45.013305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:45.013305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:45.013305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:45.017494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 278/558] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:45.019690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.053249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba79c16-15d0-4e8e-a1a3-13dcafe16055 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.091883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad30a9-c551-412b-8880-d506c8fbfeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.093073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad30a9-c551-412b-8880-d506c8fbfeb6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.093603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad30a9-c551-412b-8880-d506c8fbfeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.094200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad30a9-c551-412b-8880-d506c8fbfeb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.106288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad30a9-c551-412b-8880-d506c8fbfeb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-ebad30a9-c551-412b-8880-d506c8fbfeb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:45.114831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:45.115780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:45.116600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:45.124826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 281/559] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:45.126165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad30a9-c551-412b-8880-d506c8fbfeb6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.136295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acd7cd-b984-4b5f-b575-65c7eba32421 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.177379 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.178097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.178383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.178662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.179395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:45.179702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:45.182348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.194630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 279/560] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:45.195687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.224345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:45.239982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:45.254248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad30a9-c551-412b-8880-d506c8fbfeb6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.289305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9c709-85de-4434-aee8-91d643d84baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.290283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c709-85de-4434-aee8-91d643d84baa None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.290705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c709-85de-4434-aee8-91d643d84baa None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.291006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9c709-85de-4434-aee8-91d643d84baa None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.292037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9c709-85de-4434-aee8-91d643d84baa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:45.292352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9c709-85de-4434-aee8-91d643d84baa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:45.293896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9c709-85de-4434-aee8-91d643d84baa None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.307896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 282/561] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:45.308221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835e42-ccf5-41b8-b16b-85a64b38c2cb None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.309360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98395894-2b02-4ee5-bd3e-92a4c8f26cc8 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.309646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98395894-2b02-4ee5-bd3e-92a4c8f26cc8 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.309968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98395894-2b02-4ee5-bd3e-92a4c8f26cc8 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.311364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 283/562] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:45.315848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98395894-2b02-4ee5-bd3e-92a4c8f26cc8 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.347261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.348057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.348343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.348621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.349383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:45.349692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:45.351111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.364275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 280/563] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:45.364275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.437535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98395894-2b02-4ee5-bd3e-92a4c8f26cc8 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.478617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:45.490316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd987c0-eed4-4807-bdb2-2739b0be6950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.491179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd987c0-eed4-4807-bdb2-2739b0be6950 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.491464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd987c0-eed4-4807-bdb2-2739b0be6950 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.491740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd987c0-eed4-4807-bdb2-2739b0be6950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.495840 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b9294-7cf6-4168-beac-a141c2e2e0c1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.499086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd987c0-eed4-4807-bdb2-2739b0be6950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:45.500583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd987c0-eed4-4807-bdb2-2739b0be6950 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-9dd987c0-eed4-4807-bdb2-2739b0be6950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:45.519378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:45.520487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:45.521734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 284/564] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:45.522523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca101b9-6af3-424a-a683-7747ff98c616 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.522523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca101b9-6af3-424a-a683-7747ff98c616 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.522803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca101b9-6af3-424a-a683-7747ff98c616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.540487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-dca101b9-6af3-424a-a683-7747ff98c616 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:45.541365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.541700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-dca101b9-6af3-424a-a683-7747ff98c616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:45.542509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.542803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.543069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.543750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:45.552396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:45.554108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:45.569342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:45.570572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 15:40:45.571803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:45.577937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 281/565] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:45.578858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.614580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 285/566] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:45.615927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ad3ba-4f57-4e15-8159-262136b616d7 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.616216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ad3ba-4f57-4e15-8159-262136b616d7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.616498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ad3ba-4f57-4e15-8159-262136b616d7 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.622314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 286/567] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:45.622314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ad3ba-4f57-4e15-8159-262136b616d7 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.711017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92db4b-0473-4e70-94c7-cc0a31f99ab0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.728410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ad3ba-4f57-4e15-8159-262136b616d7 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.747635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfb164-bca2-404b-a782-90de1d4e9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.748432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb164-bca2-404b-a782-90de1d4e9c4d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.748710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb164-bca2-404b-a782-90de1d4e9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.748985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb164-bca2-404b-a782-90de1d4e9c4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.759732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfb164-bca2-404b-a782-90de1d4e9c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:45.761210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfb164-bca2-404b-a782-90de1d4e9c4d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.762740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097975c9-6898-4790-af29-e209b842d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.763813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097975c9-6898-4790-af29-e209b842d688 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.763926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097975c9-6898-4790-af29-e209b842d688 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.764121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097975c9-6898-4790-af29-e209b842d688 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.764920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097975c9-6898-4790-af29-e209b842d688 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:45.765232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097975c9-6898-4790-af29-e209b842d688 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:45.766726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097975c9-6898-4790-af29-e209b842d688 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.773902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-dfdfb164-bca2-404b-a782-90de1d4e9c4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:45.778578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:45.779516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:45.780367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:45.780367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:45.780367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:45.780367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:45.780594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 287/568] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:45.780594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097975c9-6898-4790-af29-e209b842d688 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.789019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 282/569] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:45.789019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2a4b1-d543-4ddc-9027-b004639f4880 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.789019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2a4b1-d543-4ddc-9027-b004639f4880 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.789019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2a4b1-d543-4ddc-9027-b004639f4880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.791759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:45.804539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-26b2a4b1-d543-4ddc-9027-b004639f4880 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:45.805835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-26b2a4b1-d543-4ddc-9027-b004639f4880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:45.875223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 283/570] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:45.876160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2a4b1-d543-4ddc-9027-b004639f4880 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.881456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097975c9-6898-4790-af29-e209b842d688 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.917250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:45.917972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:45.918258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:45.918535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:45.919417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:45.919728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:45.921202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:45.943559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 288/571] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:45.943559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:45.979296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2a4b1-d543-4ddc-9027-b004639f4880 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:45.999376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:46.012467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:46.017959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e609-47f6-4466-807e-824d3bcf667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.018769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e609-47f6-4466-807e-824d3bcf667a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.019048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e609-47f6-4466-807e-824d3bcf667a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.019316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5e609-47f6-4466-807e-824d3bcf667a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.030383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec5e609-47f6-4466-807e-824d3bcf667a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-1ec5e609-47f6-4466-807e-824d3bcf667a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:46.038117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:46.039317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:46.040441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:46.045111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 284/572] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:46.046102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e609-47f6-4466-807e-824d3bcf667a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.050848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493f5bf-bbab-4dc9-839b-6dc432876149 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.084922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.085591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.085905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.086181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.087092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=7230dc79eb9844f1aa618d5e1c883b8a) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:46.087399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:46.088875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:46.094349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 289/573] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:45 2022] GET /identity/v3/role_assignments?user.id=7230dc79eb9844f1aa618d5e1c883b8a&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:46.095162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.162616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5e609-47f6-4466-807e-824d3bcf667a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.211035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.211787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.212074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.212377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.213216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:46.213530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:46.215022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:46.228435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 285/574] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:46.229390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.249758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c73186-08fd-4307-a074-eba09078e10e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.280660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:46.295536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:46.298054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce7c97-2a9a-4d1c-9309-a493cf8f2b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.298931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7c97-2a9a-4d1c-9309-a493cf8f2b67 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.299212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7c97-2a9a-4d1c-9309-a493cf8f2b67 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.299685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce7c97-2a9a-4d1c-9309-a493cf8f2b67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.311444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce7c97-2a9a-4d1c-9309-a493cf8f2b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-a1ce7c97-2a9a-4d1c-9309-a493cf8f2b67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:46.319371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:46.320274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 290/575] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:46.322359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce7c97-2a9a-4d1c-9309-a493cf8f2b67 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.360005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62a349-af56-41a1-8c7b-a6d27f799e7a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.401706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.403081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.403635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.403918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.421294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:46.422810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:46.436421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce7c97-2a9a-4d1c-9309-a493cf8f2b67 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:46.444574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:46.445791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:46.452166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:46.452166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:46.452166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:46.452166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:46.452166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 286/576] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:46.452166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.475901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.476710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.477058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.477442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.478243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:46.478558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:46.480066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:46.490186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 291/577] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:46.491040 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.556252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:46.614142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55048932-4004-45d9-81f9-cd22f03c1089 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.618260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:46.623409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024fccc-7b19-41a7-83f6-b73193b13c14 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.662612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.663443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.663718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.663958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.673305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:46.673982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.674693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.674845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:46.674982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.675315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.676146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:46.676464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:46.677981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:46.692952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 292/578] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:46.692952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:46.694313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:46.695205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:46.696195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 287/579] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:46.697342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.807897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e6d98-abdc-4f45-9939-08bb07b164c1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.810297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:46.816816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdc801-1af3-4d50-adae-d6489251cd46 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:46.843563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.844232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.844510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.844780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.845615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:46.846052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:46.847440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:46.853899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:46.854715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:46.855103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:46.855439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:46.861799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 293/580] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:46.862931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.870803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:46.872306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:46.891108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:46.892024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:46.892990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:46.892990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:46.892990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:46.892990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:46.892990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 288/581] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:46.893279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:46.917733 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:46.973685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c34228b-e8e6-4cf0-89ad-48c506e47afd None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.007967 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca538fb-5c6b-4b97-aba6-d42766bae688 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.013631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.014502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.014764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.015033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.026730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.028278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:47.047007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:47.047918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:47.053641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:47.053641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:47.053641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:47.053641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:47.053900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.053900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.053900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.054124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 294/582] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:47.054201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.054276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.054352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:47.054352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.054352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.072490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:47.073173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 289/583] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:40:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:47.074785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.171930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8afbf-8549-4188-a889-c41fd9102dad None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.172613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:47.199845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c05da3-c622-4650-93e0-26766035a997 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.214382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.215255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.215583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.215828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.223413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.224888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:47.239553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:47.240474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:47.241355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 295/584] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:47.241895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.250918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.251678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.251971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.252247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.252987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:47.253810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.255230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.273060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 290/585] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:47.273060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.334111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:47.358768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5776-547c-4e70-b782-ddc3271df20b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.379071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039333e2-7385-4a7b-8ebf-47afe6d74c42 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.393204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.393908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.394186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.394458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.395189 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:47.395487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.397219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.413715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 296/586] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:47.413715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.422067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.422926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.423215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.423492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:47.436873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.438587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:47.458038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:47.458980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:47.459833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:47.459833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:47.459833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:47.459833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:47.459833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 291/587] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:47.460591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.520103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91f9e3-caf0-4ce5-b79c-9d9098dc044b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.566032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.566899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.567247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.567527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.568404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:47.568714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.570281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.573494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:47.590126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df04b-40cb-4730-bc95-1533e75e87a6 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.590919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 297/588] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:47.591832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.621887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.622692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.622979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.623250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] PATH_INFO: `/v3/users/glance` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.630924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.632382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:47.647541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:47.648539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 292/589] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:47.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.700156 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:47.705365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2f814-5835-4564-9846-9362bea5bae5 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.739805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.740610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.740896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.741169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.749531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.751037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.761233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d4e1e-1a8c-45e3-b66d-fa57db183b4b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:47.767895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:47.768811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:47.769736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 298/590] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:47.770356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.802048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.802864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.803155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.803432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.804170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:47.804473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.805944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.823267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 293/591] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:47.823267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.836566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:47.886476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa55bb-ea2d-491a-aa4d-2176dff1cd28 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.923614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f6527-c176-4fa8-a40d-823338bdaf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.924882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f6527-c176-4fa8-a40d-823338bdaf9c None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.925417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f6527-c176-4fa8-a40d-823338bdaf9c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.925926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f6527-c176-4fa8-a40d-823338bdaf9c None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/ed0934c0b1a64dfca3431dc32441e2db/roles/7a558a55cb054e62ac77df614c5bd73d` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.928052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895a1c2-2bf0-4ffc-801c-816ebcff2613 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:47.936410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:47.943585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f6527-c176-4fa8-a40d-823338bdaf9c None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=ed0934c0b1a64dfca3431dc32441e2db, role_id=7a558a55cb054e62ac77df614c5bd73d)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.945206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f6527-c176-4fa8-a40d-823338bdaf9c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:47.962133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 299/592] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:47 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/ed0934c0b1a64dfca3431dc32441e2db/roles/7a558a55cb054e62ac77df614c5bd73d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:47.963275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f6527-c176-4fa8-a40d-823338bdaf9c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:47.980235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:47.981508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:47.981924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:47.982314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:47.983527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:47.984276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:47.986635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.018290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 294/593] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:48.019363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.123384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f6527-c176-4fa8-a40d-823338bdaf9c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.135414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:48.152060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5200c4-233e-4b9e-9a19-e38463adbfc2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.173855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.174567 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.174855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.175137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.176026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:48.176339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:48.178814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.193807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437401c8-a5cf-4e0e-85f3-355acc1842ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.194631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437401c8-a5cf-4e0e-85f3-355acc1842ea None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.194916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437401c8-a5cf-4e0e-85f3-355acc1842ea None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.195194 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437401c8-a5cf-4e0e-85f3-355acc1842ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.203037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 300/594] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Jun 21 15:40:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:48.207768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437401c8-a5cf-4e0e-85f3-355acc1842ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:48.207943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.208037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437401c8-a5cf-4e0e-85f3-355acc1842ea None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-437401c8-a5cf-4e0e-85f3-355acc1842ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:48.221195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:48.222191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:48.223051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 295/595] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:48.224130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0e70-8f78-44a2-8282-032c394b038c None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.224130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0e70-8f78-44a2-8282-032c394b038c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.224294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0e70-8f78-44a2-8282-032c394b038c None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.225766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 296/596] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:48.226493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f0e70-8f78-44a2-8282-032c394b038c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.264329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:48.335511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74d218-fdc8-47c4-a9f7-933d37294a69 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.360481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f0e70-8f78-44a2-8282-032c394b038c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.374843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550bb9c-a4b3-4b87-9a8f-83baa6d09e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.375762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550bb9c-a4b3-4b87-9a8f-83baa6d09e22 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.376039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550bb9c-a4b3-4b87-9a8f-83baa6d09e22 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.376321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550bb9c-a4b3-4b87-9a8f-83baa6d09e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.387790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550bb9c-a4b3-4b87-9a8f-83baa6d09e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:48.389258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550bb9c-a4b3-4b87-9a8f-83baa6d09e22 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.395874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.397063 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.397874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.398155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-f550bb9c-a4b3-4b87-9a8f-83baa6d09e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:48.409076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:48.410202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:48.411099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:48.411099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:48.411099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:48.411099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:48.411099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 301/597] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:48.411705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33f4a-9640-4297-b734-858144407f61 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.411800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33f4a-9640-4297-b734-858144407f61 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.412047 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e33f4a-9640-4297-b734-858144407f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.414771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:48.416562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.430629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-82e33f4a-9640-4297-b734-858144407f61 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:48.431838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-82e33f4a-9640-4297-b734-858144407f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:48.437680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:48.438816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:48.439702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:48.439702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:48.439702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:48.439702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:48.439702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 297/598] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:48.440392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.448572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:48.515392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 302/599] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:48.516256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e33f4a-9640-4297-b734-858144407f61 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.528366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:48.557645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d1c7d-2b41-4bdd-82cf-6a3d457de095 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.595827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.596502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.596789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.597309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.597892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:48.598210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:48.599690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.617353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 298/600] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:48.617353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.655787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e33f4a-9640-4297-b734-858144407f61 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.692125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.693186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.693707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.694283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.695291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:48.695707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:48.697341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.714824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 303/601] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:48.715563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.726277 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:48.730005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55440298-7b26-4649-b9bb-3c952ba9fcef None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.768086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.768772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.769066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.769340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.770101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:48.770408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:48.772187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.774665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:48.789924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 299/602] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:48.789924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.841255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fd3eb-4c42-4a6f-a585-2ab043537fb7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.878954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169268a2-1e5a-494c-9da1-a571524a3237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.880498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169268a2-1e5a-494c-9da1-a571524a3237 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.880954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169268a2-1e5a-494c-9da1-a571524a3237 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.881421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169268a2-1e5a-494c-9da1-a571524a3237 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.892549 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169268a2-1e5a-494c-9da1-a571524a3237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-169268a2-1e5a-494c-9da1-a571524a3237 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:48.902332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:48.903477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:48.904324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:48.905024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b5232-2f4e-4f7b-b08c-6a14f14c8b35 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:48.906709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 304/603] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:48.907705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169268a2-1e5a-494c-9da1-a571524a3237 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:48.952469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560fde-7589-4cea-b5ea-eb6baf57d8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:48.953846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560fde-7589-4cea-b5ea-eb6baf57d8f1 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:48.954135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560fde-7589-4cea-b5ea-eb6baf57d8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:48.954412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4560fde-7589-4cea-b5ea-eb6baf57d8f1 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/46e5ca80b7834ab7bc951b454c9aae49/roles/7a558a55cb054e62ac77df614c5bd73d` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:48.972067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4560fde-7589-4cea-b5ea-eb6baf57d8f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=46e5ca80b7834ab7bc951b454c9aae49, role_id=7a558a55cb054e62ac77df614c5bd73d)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:48.973508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4560fde-7589-4cea-b5ea-eb6baf57d8f1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:48.985736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:48.993812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 300/604] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:48 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/46e5ca80b7834ab7bc951b454c9aae49/roles/7a558a55cb054e62ac77df614c5bd73d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:48.993812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560fde-7589-4cea-b5ea-eb6baf57d8f1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:49.054559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169268a2-1e5a-494c-9da1-a571524a3237 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.061109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:49.104352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.105456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.105906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.106284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.114753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.116507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:49.139514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:49.140700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:49.141595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:49.141595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:49.141595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:49.141595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:49.141595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:49.141595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:49.141595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:49.141595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:49.142074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4560fde-7589-4cea-b5ea-eb6baf57d8f1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.147399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 305/605] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:49.148447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:49.190009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.190973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.191265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.191562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.202018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.204268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:49.222873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:49.223830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:49.224690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 301/606] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:49.225164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:49.292654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d4691-fd16-44dd-a7d9-8530a0a328d2 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.320570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:49.331165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.332103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.332392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.332675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.333960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:49.334278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.336241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.348483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 306/607] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:49.349875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:49.355664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270b0eb-7678-4c83-ac78-a848e8341b7f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.364064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:49.396868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.397984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.398514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.399062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.400453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:49.401026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.402884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.418386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 302/608] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:49.419606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:49.487103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497f0c8-3d3e-4572-a000-f581a848cff5 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.539458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.540172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.540455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.540732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.541546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:49.541888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.544079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.556517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cd1bf-c792-4e19-a244-b88ec0351f59 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.562915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 307/609] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:49.563801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:49.598323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.599534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.600211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.600816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.612772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.614620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.632968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:49.636790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:49.637660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:49.638589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:49.638589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:49.638589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:49.638589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:49.640996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 303/610] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:49.642213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:49.673818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:49.690370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86ca6b-8de0-4d93-96d2-e8f3a06d9f5b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.729962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435d16-1881-4928-8b1a-ec341a247ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.731288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435d16-1881-4928-8b1a-ec341a247ff3 None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.731777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435d16-1881-4928-8b1a-ec341a247ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.732267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d435d16-1881-4928-8b1a-ec341a247ff3 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/5cb5dc4c5a5a448294c57a2db5938d9d/roles/7a558a55cb054e62ac77df614c5bd73d` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.750106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d435d16-1881-4928-8b1a-ec341a247ff3 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=5cb5dc4c5a5a448294c57a2db5938d9d, role_id=7a558a55cb054e62ac77df614c5bd73d)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.751967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d435d16-1881-4928-8b1a-ec341a247ff3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.772806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39e456-516f-4eaf-a26a-b823ccf8e5df None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.774174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 308/611] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:49 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/5cb5dc4c5a5a448294c57a2db5938d9d/roles/7a558a55cb054e62ac77df614c5bd73d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:49.775501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435d16-1881-4928-8b1a-ec341a247ff3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:49.823818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb18d5-8b85-4aca-aec2-0a7910ba3d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.825204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb18d5-8b85-4aca-aec2-0a7910ba3d24 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.825706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb18d5-8b85-4aca-aec2-0a7910ba3d24 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.826206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb18d5-8b85-4aca-aec2-0a7910ba3d24 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/6efa4bed025741d492158b5c6e45c7e7/roles/7a558a55cb054e62ac77df614c5bd73d` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.844378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb18d5-8b85-4aca-aec2-0a7910ba3d24 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=6efa4bed025741d492158b5c6e45c7e7, role_id=7a558a55cb054e62ac77df614c5bd73d)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.846216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb18d5-8b85-4aca-aec2-0a7910ba3d24 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.863184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 304/612] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:49 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/6efa4bed025741d492158b5c6e45c7e7/roles/7a558a55cb054e62ac77df614c5bd73d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:49.915760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:49.919923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d435d16-1881-4928-8b1a-ec341a247ff3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:49.962174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48a5c6-4a09-46e9-9cd3-94c19808ec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:49.962947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48a5c6-4a09-46e9-9cd3-94c19808ec8b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:49.963235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48a5c6-4a09-46e9-9cd3-94c19808ec8b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:49.963514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48a5c6-4a09-46e9-9cd3-94c19808ec8b None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:49.964328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48a5c6-4a09-46e9-9cd3-94c19808ec8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:49.964663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48a5c6-4a09-46e9-9cd3-94c19808ec8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:49.966335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48a5c6-4a09-46e9-9cd3-94c19808ec8b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:49.980594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 309/613] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:49.985998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb18d5-8b85-4aca-aec2-0a7910ba3d24 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:50.072801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:50.114691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb18d5-8b85-4aca-aec2-0a7910ba3d24 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:50.124204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08140b0b-1574-447a-b411-89fbdcf7c8e4 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:50.124839 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08140b0b-1574-447a-b411-89fbdcf7c8e4 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:50.125122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08140b0b-1574-447a-b411-89fbdcf7c8e4 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:50.126833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 310/614] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:50.133120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9920f-be33-461b-85a8-be9d6d30757a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:50.133645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9920f-be33-461b-85a8-be9d6d30757a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:50.133755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9920f-be33-461b-85a8-be9d6d30757a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:50.159217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:50.160290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:50.160719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:50.161218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:50.170957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-bde9920f-be33-461b-85a8-be9d6d30757a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:50.171641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:50.172211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bde9920f-be33-461b-85a8-be9d6d30757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:50.173126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:50.188958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:50.190299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:40:50.191245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:50.193430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 305/615] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:50.197033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:50.289797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 311/616] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:50.318145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771ffcd-9903-4584-b60a-34fc7da3bbe3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:50.329288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9920f-be33-461b-85a8-be9d6d30757a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:50.385128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:50.406063 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:50.406757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:50.407037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:50.407307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:50.408153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:50.408460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:50.410163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:50.435008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 306/617] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jun 21 15:40:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:50.447578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:50.502883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9920f-be33-461b-85a8-be9d6d30757a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:50.506101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:50.562619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9e28c-516d-4cad-ac08-e96e76985434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:50.563450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9e28c-516d-4cad-ac08-e96e76985434 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:50.563736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9e28c-516d-4cad-ac08-e96e76985434 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:50.564016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9e28c-516d-4cad-ac08-e96e76985434 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:50.575460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca9e28c-516d-4cad-ac08-e96e76985434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-6ca9e28c-516d-4cad-ac08-e96e76985434 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:50.587954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:50.588936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:50.590053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 312/618] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:40:50.598832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9e28c-516d-4cad-ac08-e96e76985434 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:50.604686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111e2c6-b599-41cc-8a6f-31213d8ab596 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:50.671701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:50.680717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:50.680717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:50.680717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:50.708133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:50.710413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:50.718345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9e28c-516d-4cad-ac08-e96e76985434 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:50.746660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:50.747705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:50.748757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:50.748757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:50.748757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:50.748757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:50.748757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 307/619] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:50.759035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:50.780450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:50.781480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:50.781961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:50.782418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:50.783398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:50.783881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:50.785559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:50.802935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 313/620] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:50.811203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:50.827810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:50.858229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:50.959448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8809c-72cd-42da-817e-dd71e18a43c4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:50.996496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:50.997311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:50.997590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:50.997909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.009967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:51.011538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:51.032050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:51.033019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:51.033925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:51.033925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:51.033925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:51.033925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:51.033925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 314/621] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:51.038961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:51.059937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719f691-9c97-4197-86b5-d046c676024c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:51.110253 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:51.153358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41149b30-9cda-4183-aa7e-56c40793e8ce None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:51.191784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:51.192474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.192758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.193036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.193487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:51.193789 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:51.194109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:51.194315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.194602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.194882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.195644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:51.195809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:51.196141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:51.197696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:51.223245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 315/622] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:51.229841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:51.241800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 308/623] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:51.251923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:51.380885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33836c-4218-44be-878f-f7b778e86b86 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:51.414392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:51.426039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:51.426860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.427146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.427426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.436015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:51.437534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:51.453879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:51.454915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:51.455850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 316/624] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:51.461264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:51.513142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:51.524232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9aa3b-40b5-424f-9751-f5324ef559e8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:51.574645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c83c7-670c-4ef2-83b0-6fa5ab1addfb None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:51.622109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86cc89-9edb-482b-b47e-66461b3e1019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:51.623083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cc89-9edb-482b-b47e-66461b3e1019 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.623370 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cc89-9edb-482b-b47e-66461b3e1019 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.623653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cc89-9edb-482b-b47e-66461b3e1019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.624536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d20e37-e8ef-4c35-b2a5-6686594aad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:51.625237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d20e37-e8ef-4c35-b2a5-6686594aad59 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.625523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d20e37-e8ef-4c35-b2a5-6686594aad59 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.625859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d20e37-e8ef-4c35-b2a5-6686594aad59 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.627204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d20e37-e8ef-4c35-b2a5-6686594aad59 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:51.627803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d20e37-e8ef-4c35-b2a5-6686594aad59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:51.629759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d20e37-e8ef-4c35-b2a5-6686594aad59 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:51.635918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86cc89-9edb-482b-b47e-66461b3e1019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:51.637308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e86cc89-9edb-482b-b47e-66461b3e1019 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:51.650311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 317/625] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jun 21 15:40:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:51.651686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab98b6-55a0-4947-8e37-f57e59f343f1 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.651975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab98b6-55a0-4947-8e37-f57e59f343f1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.652254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab98b6-55a0-4947-8e37-f57e59f343f1 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-1e86cc89-9edb-482b-b47e-66461b3e1019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:51.653176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:51.654056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:51.654944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:51.654944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:51.654944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:51.654944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:51.654944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:51.654944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:51.654944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:51.654944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:51.655386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 318/626] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:51.657175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 309/627] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:51.659864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab98b6-55a0-4947-8e37-f57e59f343f1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:51.663294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240920b-80e2-464a-8792-45117e4f976d None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.663833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240920b-80e2-464a-8792-45117e4f976d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.664322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240920b-80e2-464a-8792-45117e4f976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.690866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-9240920b-80e2-464a-8792-45117e4f976d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:51.692812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9240920b-80e2-464a-8792-45117e4f976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:51.712811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:51.767168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 310/628] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:51.768184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240920b-80e2-464a-8792-45117e4f976d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:51.785551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab98b6-55a0-4947-8e37-f57e59f343f1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:51.826356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db062b-3242-45a2-900d-6510dac2d844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:51.827027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db062b-3242-45a2-900d-6510dac2d844 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.827351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db062b-3242-45a2-900d-6510dac2d844 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.827622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db062b-3242-45a2-900d-6510dac2d844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.862837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:51.877003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240920b-80e2-464a-8792-45117e4f976d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:51.912127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db062b-3242-45a2-900d-6510dac2d844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:51.913439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:51.914233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:51.914471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db062b-3242-45a2-900d-6510dac2d844 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:51.914657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:51.914914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:51.915565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:51.915874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:51.917335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:51.934908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 311/629] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:51.935885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:51.963909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:52.033607 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 319/630] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:52.034904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db062b-3242-45a2-900d-6510dac2d844 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:52.045608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cbf43-4721-40a9-9361-a9180c2a2a11 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:52.082112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc88732-4fe1-4626-b64d-f08c243cd198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:52.083297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc88732-4fe1-4626-b64d-f08c243cd198 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.083586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc88732-4fe1-4626-b64d-f08c243cd198 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.083869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc88732-4fe1-4626-b64d-f08c243cd198 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.096362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc88732-4fe1-4626-b64d-f08c243cd198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-edc88732-4fe1-4626-b64d-f08c243cd198 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:52.104299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:52.105274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:52.106211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 312/631] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:52.106915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc88732-4fe1-4626-b64d-f08c243cd198 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:52.119422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:52.159462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db062b-3242-45a2-900d-6510dac2d844 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:52.202010 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5e35-b1b0-4e22-854a-893a782bebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:52.203428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5e35-b1b0-4e22-854a-893a782bebea None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.203921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5e35-b1b0-4e22-854a-893a782bebea None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.204393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc5e35-b1b0-4e22-854a-893a782bebea None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/7230dc79eb9844f1aa618d5e1c883b8a/roles/7a558a55cb054e62ac77df614c5bd73d` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.218523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:52.219429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc88732-4fe1-4626-b64d-f08c243cd198 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:52.224447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5e35-b1b0-4e22-854a-893a782bebea None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=7230dc79eb9844f1aa618d5e1c883b8a, role_id=7a558a55cb054e62ac77df614c5bd73d)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:52.226841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc5e35-b1b0-4e22-854a-893a782bebea None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:52.251011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 320/632] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:40:52 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/7230dc79eb9844f1aa618d5e1c883b8a/roles/7a558a55cb054e62ac77df614c5bd73d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:40:52.252291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5e35-b1b0-4e22-854a-893a782bebea None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:52.281921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:52.282886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.283174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.283455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.305015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:52.306641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:52.335715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:52.337036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:52.337982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:52.337982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:52.337982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:52.337982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:52.343013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 313/633] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:52.348280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:52.386803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc5e35-b1b0-4e22-854a-893a782bebea None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:52.434285 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3da1d-ad9f-4e65-8e8a-81d7cc80dbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:52.435099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3da1d-ad9f-4e65-8e8a-81d7cc80dbe5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.435384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3da1d-ad9f-4e65-8e8a-81d7cc80dbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.435656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3da1d-ad9f-4e65-8e8a-81d7cc80dbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.436437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3da1d-ad9f-4e65-8e8a-81d7cc80dbe5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:52.436815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3da1d-ad9f-4e65-8e8a-81d7cc80dbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:52.438735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3da1d-ad9f-4e65-8e8a-81d7cc80dbe5 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:52.447308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:52.453949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 321/634] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:52.453949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9eb72-0390-46bc-af4d-2ce769260800 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.453949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9eb72-0390-46bc-af4d-2ce769260800 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.453949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9eb72-0390-46bc-af4d-2ce769260800 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.458843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 322/635] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:52.458843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1ce62-202e-4ee4-9401-98c3d0030408 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.458843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1ce62-202e-4ee4-9401-98c3d0030408 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.458843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1ce62-202e-4ee4-9401-98c3d0030408 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.458843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 323/636] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:52.458843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1ce62-202e-4ee4-9401-98c3d0030408 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:52.514129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98695391-3de6-44d8-a7fc-0a5d06235665 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:52.530764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:52.555223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44e03b-9990-4bfd-a071-58da16705dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:52.555907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44e03b-9990-4bfd-a071-58da16705dc5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.556183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44e03b-9990-4bfd-a071-58da16705dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.556456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44e03b-9990-4bfd-a071-58da16705dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.557182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44e03b-9990-4bfd-a071-58da16705dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:52.557487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44e03b-9990-4bfd-a071-58da16705dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:52.558901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44e03b-9990-4bfd-a071-58da16705dc5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:52.571761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 314/637] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:52.573469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ecef4-1473-43db-a990-1c28e90dce61 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.573726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ecef4-1473-43db-a990-1c28e90dce61 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.578835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ecef4-1473-43db-a990-1c28e90dce61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.589856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1ce62-202e-4ee4-9401-98c3d0030408 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:52.596648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-650ecef4-1473-43db-a990-1c28e90dce61 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:52.597899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-650ecef4-1473-43db-a990-1c28e90dce61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:52.634045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad3c3e-01ba-4af2-b74b-b334a8444bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:52.635009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3c3e-01ba-4af2-b74b-b334a8444bf0 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.635412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3c3e-01ba-4af2-b74b-b334a8444bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.635993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3c3e-01ba-4af2-b74b-b334a8444bf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.650620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad3c3e-01ba-4af2-b74b-b334a8444bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:52.652085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad3c3e-01ba-4af2-b74b-b334a8444bf0 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-c4ad3c3e-01ba-4af2-b74b-b334a8444bf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:52.670075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:52.671052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:52.671932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:52.671932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:52.671932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:52.671932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:52.671932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 324/638] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:52.672534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037d552-6f9c-4afb-a198-7a376b271ab7 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.672823 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037d552-6f9c-4afb-a198-7a376b271ab7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.673103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037d552-6f9c-4afb-a198-7a376b271ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.686071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 315/639] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:52.687503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ecef4-1473-43db-a990-1c28e90dce61 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:52.690462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a037d552-6f9c-4afb-a198-7a376b271ab7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:52.691664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a037d552-6f9c-4afb-a198-7a376b271ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:52.755733 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:52.767425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 325/640] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:52.768370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037d552-6f9c-4afb-a198-7a376b271ab7 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:52.813023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:52.823292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ecef4-1473-43db-a990-1c28e90dce61 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:52.864744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:52.865832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.865832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.866121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.874066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:52.875553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:52.891112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:52.892102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:52.892979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 316/641] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:52.894364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:52.926257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037d552-6f9c-4afb-a198-7a376b271ab7 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:52.978677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:52.979466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:52.980031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:52.980362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:52.981180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:52.981556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:52.983152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:52.997933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 326/642] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:52.998791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.037351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f4d3d-6f37-4873-ac95-d864a0495d13 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.059023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:53.098945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55d28-7305-42df-aee4-3fb119f3ae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.100288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55d28-7305-42df-aee4-3fb119f3ae78 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.100779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55d28-7305-42df-aee4-3fb119f3ae78 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.101248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55d28-7305-42df-aee4-3fb119f3ae78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.104406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:53.112167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff6c57-807a-4269-bd0d-1e556bad7ffc None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.120626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55d28-7305-42df-aee4-3fb119f3ae78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-e5c55d28-7305-42df-aee4-3fb119f3ae78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:53.135610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:53.136798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:53.137827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 317/643] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:53.139145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55d28-7305-42df-aee4-3fb119f3ae78 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.158218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b05d-16e5-452c-8304-84db41e984ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.159042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b05d-16e5-452c-8304-84db41e984ea None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.159325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b05d-16e5-452c-8304-84db41e984ea None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.159601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b05d-16e5-452c-8304-84db41e984ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.170551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3b05d-16e5-452c-8304-84db41e984ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-3fa3b05d-16e5-452c-8304-84db41e984ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:53.178328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:53.179469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 327/644] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:53.185917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b05d-16e5-452c-8304-84db41e984ea None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.267665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55d28-7305-42df-aee4-3fb119f3ae78 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.303906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.304492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.304805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.304991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b05d-16e5-452c-8304-84db41e984ea None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.305133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.305909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:53.306222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:53.307705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:53.321392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 318/645] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:53.322327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.350516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.351821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.352116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.352407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.355457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:53.362157 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:53.363722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:53.379533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:53.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 328/646] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:53.386256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.403821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:53.445102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65745cf-bb17-480a-93e3-bd0f646cfb61 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.485793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.486486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.486769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.487044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.487832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] RBAC: Adding query filter params (name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:53.488112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:53.489542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:53.499822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 319/647] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:53.500668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.508333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07716d3d-7b2c-475f-82ba-3d53cf92785e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.547725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.548865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.549409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.549941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.551004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:53.551610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:53.553357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:53.564145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 329/648] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:53.565431 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.608655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:53.619892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04d42a-28a2-404d-ae65-51345f8bf124 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.661101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.661790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.662095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.662368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.662902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:53.663197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=ed0934c0b1a64dfca3431dc32441e2db) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:53.663508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:53.665098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:53.672849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 320/649] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/role_assignments?user.id=ed0934c0b1a64dfca3431dc32441e2db&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:53.674046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.705274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cb72b-e578-4988-9a88-5c6f20a02140 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.754849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.755757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.756111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.756462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.757547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:53.758397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:53.760116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:53.787526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 330/650] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:53.788682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.805745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbf7a1-cf73-41fd-a2fb-0b16190baae2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.849344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.850523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.850985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.852316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.865414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:53.868225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:53.886911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:53.887856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:53.888725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:53.888725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:53.888725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:53.888725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:53.888725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 321/651] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:53.889450 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:53.908460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:53.929097 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682ea6-ab91-483a-a3da-1193bd1a03da None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:53.982205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:53.983779 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:53.984212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:53.984633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:53.987261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:54.004922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.007413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:54.033555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:54.034717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:54.035768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:54.035768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:54.035768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:54.035768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:54.038413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 331/652] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:54.039702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.042390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062fc88-2853-49d2-bd47-4790d7d04caa None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:54.087677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8651-184f-4f03-b6f5-8e8399b144e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:54.088641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8651-184f-4f03-b6f5-8e8399b144e0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.089697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8651-184f-4f03-b6f5-8e8399b144e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.090190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8651-184f-4f03-b6f5-8e8399b144e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.172669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687167d2-6e48-4db3-afc7-4854e3629710 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:54.172965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a8651-184f-4f03-b6f5-8e8399b144e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.174727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a8651-184f-4f03-b6f5-8e8399b144e0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.186113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:54.218929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:54.219876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.220328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.220820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.221886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:54.222391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.224002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.236844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 332/653] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:54.238103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.302209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:54.312840 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 322/654] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:54.314415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8651-184f-4f03-b6f5-8e8399b144e0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.358762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e77142-b604-43b6-a366-144ba73bfa2b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:54.396902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:54.397753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.398183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.399046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.399973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:54.400668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.402018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.420110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 333/655] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:54.420110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.434499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8651-184f-4f03-b6f5-8e8399b144e0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:54.446611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:54.473183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:54.474030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.474322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.474584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.483561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.485041 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:54.501881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:54.503095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 21 15:40:54.504219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:54.507658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 323/656] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:54.508567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.547594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278e61b-5e40-47de-8a2f-1f9e6ea772a4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:54.555776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:54.586901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d9d38-a3d8-4ce7-b2de-846d68689b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:54.587728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d9d38-a3d8-4ce7-b2de-846d68689b7b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.588019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d9d38-a3d8-4ce7-b2de-846d68689b7b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.588307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d9d38-a3d8-4ce7-b2de-846d68689b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.600297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d9d38-a3d8-4ce7-b2de-846d68689b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.602213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d9d38-a3d8-4ce7-b2de-846d68689b7b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-a53d9d38-a3d8-4ce7-b2de-846d68689b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:54.621538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:54.622536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:54.623398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:54.623398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:54.623398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:54.623398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:54.623398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 334/657] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:54.624589 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53b6e9-afd7-4aa0-ad6b-6953c1cf5585 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.624868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53b6e9-afd7-4aa0-ad6b-6953c1cf5585 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.625141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53b6e9-afd7-4aa0-ad6b-6953c1cf5585 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.630029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 335/658] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:54.630029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53b6e9-afd7-4aa0-ad6b-6953c1cf5585 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.630589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f7aa7-cc72-404e-9cd6-3f3f8314fb84 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:54.672335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:54.673157 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.673443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.673718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.684465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.686786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:54.704344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:54.705385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 21 15:40:54.706385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:54.710378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 324/659] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:54.711499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.742343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:54.758950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53b6e9-afd7-4aa0-ad6b-6953c1cf5585 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:54.798269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66ee19-7447-4fb2-b5f0-7c0ec6084b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:54.798953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66ee19-7447-4fb2-b5f0-7c0ec6084b0b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.799235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66ee19-7447-4fb2-b5f0-7c0ec6084b0b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.799509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66ee19-7447-4fb2-b5f0-7c0ec6084b0b None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.800363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66ee19-7447-4fb2-b5f0-7c0ec6084b0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:54.800669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66ee19-7447-4fb2-b5f0-7c0ec6084b0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.802149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66ee19-7447-4fb2-b5f0-7c0ec6084b0b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.826845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 336/660] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:54.826845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04437328-23c0-4fdf-85cc-1f0dc104dbbb None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.826845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04437328-23c0-4fdf-85cc-1f0dc104dbbb None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.826845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04437328-23c0-4fdf-85cc-1f0dc104dbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.831535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74bc32-1822-4dd9-b810-77312e3dfd7a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:54.848305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-04437328-23c0-4fdf-85cc-1f0dc104dbbb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:54.849542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-04437328-23c0-4fdf-85cc-1f0dc104dbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:54.862026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:54.870130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:54.870904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:54.871192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:54.871467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:54.872202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:54.872509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:54.873966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:54.890729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 325/661] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:54.890729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.934069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 337/662] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:54.934069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04437328-23c0-4fdf-85cc-1f0dc104dbbb None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:54.994634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:55.009306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b4f70-a79d-402b-9007-f120b0eac71b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.047399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04437328-23c0-4fdf-85cc-1f0dc104dbbb None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.049679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.050402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.050687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.050959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.051813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:55.052124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:55.053575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:55.073183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 326/663] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:55.073900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:55.091396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e32380-4088-4d24-ae88-988461971ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.092091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e32380-4088-4d24-ae88-988461971ef8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.092377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e32380-4088-4d24-ae88-988461971ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.092660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e32380-4088-4d24-ae88-988461971ef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.112523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:55.172463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e32380-4088-4d24-ae88-988461971ef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:55.174006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e32380-4088-4d24-ae88-988461971ef8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:55.192022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e965d1-7e74-4566-a122-b3d880fc9de5 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.231876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.232845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.233125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.233397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.240741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:55.242293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:55.257164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:55.258341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 327/664] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:55.264887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:55.274096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:55.296905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 338/665] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:55.296905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e32380-4088-4d24-ae88-988461971ef8 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:55.345636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:55.382300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1e1da-7803-41a0-a83c-d639b526354c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.407260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e32380-4088-4d24-ae88-988461971ef8 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.418929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9317cb-24dc-4936-8283-69cba98666e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.419862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9317cb-24dc-4936-8283-69cba98666e0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.420148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9317cb-24dc-4936-8283-69cba98666e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.420397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9317cb-24dc-4936-8283-69cba98666e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.432734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9317cb-24dc-4936-8283-69cba98666e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-2d9317cb-24dc-4936-8283-69cba98666e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:55.440565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:55.441526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:40:55.442431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:55.447015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 328/666] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:55.447796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9317cb-24dc-4936-8283-69cba98666e0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:55.450434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b881f-9904-4f73-ab29-87bdede2c0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.451111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b881f-9904-4f73-ab29-87bdede2c0d7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.451614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b881f-9904-4f73-ab29-87bdede2c0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.451906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b881f-9904-4f73-ab29-87bdede2c0d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.531417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:55.536908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b881f-9904-4f73-ab29-87bdede2c0d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:55.538461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b881f-9904-4f73-ab29-87bdede2c0d7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:55.573925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9317cb-24dc-4936-8283-69cba98666e0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.590807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:55.620805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.621479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.621757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.622057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.622853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:55.623162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:55.624609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:55.639849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 329/667] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:55.641047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:55.658757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 339/668] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:55.659447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b881f-9904-4f73-ab29-87bdede2c0d7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:55.757754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fe559-47d9-4bef-913d-ba0498dce7d1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.767924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b881f-9904-4f73-ab29-87bdede2c0d7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.788342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:55.798963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.799780 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.800062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.800338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.805172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.805898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.806212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.806466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.807336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:55.808062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:55.809895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:55.811443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:55.812901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:55.823987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 340/669] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:55.823987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:55.834660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:55.837381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:55.838306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:55.839168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:55.839168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:55.839168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:55.839168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:55.839168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 330/670] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:55.839469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:55.940489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4b9e4-5a13-47f0-9c2c-f3c6202bf1dd None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.960633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f247c-0672-4b60-9821-fea6227c3f8d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:55.977949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:55.978643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:55.978924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:55.979200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:55.980079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=46e5ca80b7834ab7bc951b454c9aae49) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:55.980394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:55.981947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:55.997815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 341/671] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/role_assignments?user.id=46e5ca80b7834ab7bc951b454c9aae49&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:55.997815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.024564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.025920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.026263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.026668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.041692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.043447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:56.066819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:56.067820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:56.068710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:56.068710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:56.068710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:56.068710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:56.074125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 331/672] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:56.074858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.093963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:56.113289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:56.137428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff923-db6e-4992-bdfd-871ef8105c00 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.181690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3192a-e920-4371-8ebf-473b0e796942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.182936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3192a-e920-4371-8ebf-473b0e796942 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.183236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3192a-e920-4371-8ebf-473b0e796942 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.183634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3192a-e920-4371-8ebf-473b0e796942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.199997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3192a-e920-4371-8ebf-473b0e796942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.201514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3192a-e920-4371-8ebf-473b0e796942 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.208844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d34836-177b-42f5-b23f-e0e389a695ad None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-08c3192a-e920-4371-8ebf-473b0e796942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:56.220146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:56.221148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:56.222079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:56.222079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:56.222079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:56.222079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:56.222079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 342/673] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:56.223226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a57f-0619-4140-b157-6f07fc65023f None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.223672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a57f-0619-4140-b157-6f07fc65023f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.224214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a57f-0619-4140-b157-6f07fc65023f None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.226288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 343/674] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:40:56.227644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a57f-0619-4140-b157-6f07fc65023f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.273258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.273988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.274271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.274597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.275276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:56.275589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.277013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.292111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 332/675] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:56.293013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.359013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a57f-0619-4140-b157-6f07fc65023f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.382764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:56.390692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:56.397200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b48ce3-1d5e-455d-accc-9d89b15dc496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.397945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b48ce3-1d5e-455d-accc-9d89b15dc496 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.398559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b48ce3-1d5e-455d-accc-9d89b15dc496 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.398559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b48ce3-1d5e-455d-accc-9d89b15dc496 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.399447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b48ce3-1d5e-455d-accc-9d89b15dc496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:56.399761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b48ce3-1d5e-455d-accc-9d89b15dc496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.401158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b48ce3-1d5e-455d-accc-9d89b15dc496 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.403004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959f9b-5c09-4283-a342-851a32e238d9 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.413695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 344/676] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:56.419234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd5ad0-a2d4-49ce-a807-5ac95f6b892b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.419521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd5ad0-a2d4-49ce-a807-5ac95f6b892b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.419799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd5ad0-a2d4-49ce-a807-5ac95f6b892b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.437102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f3fd5ad0-a2d4-49ce-a807-5ac95f6b892b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:56.438355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f3fd5ad0-a2d4-49ce-a807-5ac95f6b892b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:56.446505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.447188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.447475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.447753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.448492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:56.448891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.450307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.467071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 333/677] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:56.467071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.517638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 345/678] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:40:56.519303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd5ad0-a2d4-49ce-a807-5ac95f6b892b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.580210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584b3c0-222b-44ac-bf3f-0a02a6e0de33 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.618881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.620023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.620299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.620567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.628243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.629723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.638653 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd5ad0-a2d4-49ce-a807-5ac95f6b892b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:56.644215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:56.645090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:56.646643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 334/679] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:56.647121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.657472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:56.661131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:56.684529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.685581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.686057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.686721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.696495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.698661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:56.713877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:56.714791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:56.715734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 346/680] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:56.716236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.761196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00930281-8fa5-4ff4-8b08-160f20bd7520 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.798554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.800144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.800631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.801166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] PATH_INFO: `/v3/users/swift` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.812101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.814351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.826016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562e5f1-55bb-4452-b586-7f05018292e7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:56.833357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:56.834401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 335/681] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:56.841403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.863281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.864209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.864559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.864978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.865944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.867579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:56.882742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:56.883909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:40:56.884942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:56.888930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 347/682] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:56.889868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:56.915859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:56.933525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:56.950961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7891f-887d-4c24-85c1-f88eefd4768b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:56.985114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:56.985802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:56.986088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:56.986368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:56.987162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:56.987472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:56.988922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:56.998464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5cb9e-cbbd-4f2c-9c0f-4dbabe1bcd07 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:57.002722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 336/683] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:57.003635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:57.035250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:57.036282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:57.036814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:57.037235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:57.038246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:57.038843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:57.040844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:57.059774 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 348/684] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:57.059774 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:57.124437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ebf64-792a-4c63-841a-35303944c734 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:57.156232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:57.162342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:57.163032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:57.163315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:57.163592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:57.164782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:57.165224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:57.166552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:57.167627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:57.173238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0d1fa-ec9b-43ef-ab7f-bff260057e15 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:57.191641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 337/685] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jun 21 15:40:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:57.191641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:57.222286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:57.223106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:57.223454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:57.223829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:57.224653 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:57.225122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:57.226647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:57.232601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 349/686] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jun 21 15:40:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:57.233306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:57.309076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a622cd-445e-4fd6-936b-25cea091b323 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:57.340443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc337b40-4d21-47ac-a01c-69bed6c6afb1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:57.345309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:57.346015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:57.346302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:57.346578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:57.347427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=5cb5dc4c5a5a448294c57a2db5938d9d) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:57.347736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:57.349258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:57.355879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 338/687] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:57 2022] GET /identity/v3/role_assignments?user.id=5cb5dc4c5a5a448294c57a2db5938d9d&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:57.356489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:57.397504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:57.398149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:57.398513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:57.398791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:57.399651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=6efa4bed025741d492158b5c6e45c7e7) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:57.399961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:57.401443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:57.414290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 350/688] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:57 2022] GET /identity/v3/role_assignments?user.id=6efa4bed025741d492158b5c6e45c7e7&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:57.415329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:57.429809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:57.495986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:57.499496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32899c0-d630-4c1d-81c4-036d5c325fb4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:57.549656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780985be-cdbf-4775-92b1-bfc8466bcfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:57.550954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780985be-cdbf-4775-92b1-bfc8466bcfe3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:57.551773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780985be-cdbf-4775-92b1-bfc8466bcfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:57.552260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780985be-cdbf-4775-92b1-bfc8466bcfe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:57.630306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0a9b0-623c-4f48-8055-08fd06548771 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:57.658684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780985be-cdbf-4775-92b1-bfc8466bcfe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:57.660289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780985be-cdbf-4775-92b1-bfc8466bcfe3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:57.685396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:57.686579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:57.686863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:57.687140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:57.687867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:57.688178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:57.689587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:57.699776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 351/689] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jun 21 15:40:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:57.734196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:57.797044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 339/690] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:40:57.805886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:57.931019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:57.941112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60b61-ddf1-43e9-a20d-cd84891847a0 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:57.979134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:57.979960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:57.980245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:57.980518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:57.992923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:57.994384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:58.014038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:58.015538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:58.016504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:58.016504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:58.016504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:58.016504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:58.020994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 352/691] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:58.022495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780985be-cdbf-4775-92b1-bfc8466bcfe3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:58.167846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780985be-cdbf-4775-92b1-bfc8466bcfe3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:58.219398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681683f-81f4-4dd6-b33c-f0b58bdfff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:58.220086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681683f-81f4-4dd6-b33c-f0b58bdfff74 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:58.220373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681683f-81f4-4dd6-b33c-f0b58bdfff74 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:58.220651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681683f-81f4-4dd6-b33c-f0b58bdfff74 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:58.221395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681683f-81f4-4dd6-b33c-f0b58bdfff74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:58.221706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681683f-81f4-4dd6-b33c-f0b58bdfff74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:58.223121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681683f-81f4-4dd6-b33c-f0b58bdfff74 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:58.249165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 340/692] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:58.252034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:58.299209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38db1e-5ee2-4778-897a-e164362dbcfe None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:58.299537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38db1e-5ee2-4778-897a-e164362dbcfe None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:58.299779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38db1e-5ee2-4778-897a-e164362dbcfe None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:58.301162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 341/693] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:40:58.306760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b679b1c-b09d-44a9-aac2-2e9278f22e2a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:58.307051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b679b1c-b09d-44a9-aac2-2e9278f22e2a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:58.307332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b679b1c-b09d-44a9-aac2-2e9278f22e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:58.325557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2b679b1c-b09d-44a9-aac2-2e9278f22e2a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:40:58.326799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2b679b1c-b09d-44a9-aac2-2e9278f22e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:40:58.334825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:58.416236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 342/694] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:40:58.448554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36037cea-2780-43ee-a476-e3f92eb0646c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:58.483278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b679b1c-b09d-44a9-aac2-2e9278f22e2a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:58.522929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:58.524436 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:58.524715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:58.524986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:58.541452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:58.545236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:40:58.567663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:58.568618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:58.569507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:40:58.569507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:40:58.569507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:40:58.569507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:40:58.569507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:40:58.569507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:40:58.569507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:40:58.569507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:40:58.578272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 353/695] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:40:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:58.583993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:58.590135 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:58.624933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b679b1c-b09d-44a9-aac2-2e9278f22e2a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:58.666512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f9658-5a1b-4ed8-884f-75c182d0db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:58.667317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9658-5a1b-4ed8-884f-75c182d0db80 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:58.667596 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9658-5a1b-4ed8-884f-75c182d0db80 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:58.667868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9658-5a1b-4ed8-884f-75c182d0db80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:58.687211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f9658-5a1b-4ed8-884f-75c182d0db80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-ae9f9658-5a1b-4ed8-884f-75c182d0db80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:40:58.695150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:58.696220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:40:58.697072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:58.697703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 343/696] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:58.703562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f9658-5a1b-4ed8-884f-75c182d0db80 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:58.706465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:58.791352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d2767-4d66-4c0e-be3f-7358412f9553 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:58.856014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f9658-5a1b-4ed8-884f-75c182d0db80 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:58.878916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:58.879625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:58.879905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:58.880177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:58.881004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:58.881315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:58.883150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:58.923139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:58.923830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:58.924116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:58.924392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:58.925155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:58.925460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:58.927039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:58.928564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 354/697] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:40:58.929646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:58.947340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 344/698] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:40:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:58.955584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:59.009753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:59.074988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a57804-4732-4f86-b846-532bf3884ed7 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:59.098375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:59.156630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:59.157356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:59.157637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:59.157943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:59.158792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=7230dc79eb9844f1aa618d5e1c883b8a) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:59.159153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:59.160695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:59.168490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 355/699] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:40:58 2022] GET /identity/v3/role_assignments?user.id=7230dc79eb9844f1aa618d5e1c883b8a&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:59.199148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06766991-a9d9-49f8-9eea-677180ae400c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:59.268181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:59.269020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:59.269302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:59.269581 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:59.288880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:59.292095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:40:59.317905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:59.318862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:59.319773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:40:59.319773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:40:59.319773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:40:59.319773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:59.323353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 345/700] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:59.335054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:59.473641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0d81a-c6df-473b-a120-92127cefafcd None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:59.506333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:59.514840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:59.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:59.515906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:59.516601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:59.517350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:59.518048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:59.520724 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:59.535338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 356/701] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:59.541380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:59.679764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f6062-c949-4151-88fc-d5d0b270c406 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:59.703962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:40:59.725024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:59.726210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:59.726789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:59.727469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:59.736261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:59.738012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:40:59.753798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:40:59.754790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:40:59.755794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:40:59.758620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 346/702] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:40:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:40:59.764359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:59.875237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f6ae2b-a4d1-46de-9338-41e4a4846c02 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:40:59.912917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925205bd-8cb4-4276-b36d-232d656330d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:40:59.914100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925205bd-8cb4-4276-b36d-232d656330d3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:59.914743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925205bd-8cb4-4276-b36d-232d656330d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:59.915212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925205bd-8cb4-4276-b36d-232d656330d3 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:59.916434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925205bd-8cb4-4276-b36d-232d656330d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:40:59.916885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925205bd-8cb4-4276-b36d-232d656330d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:40:59.918511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925205bd-8cb4-4276-b36d-232d656330d3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:40:59.940291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 357/703] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:40:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:40:59.943857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:40:59.961615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8951d37-0458-4cfe-86ef-ef5ea5b47a3f None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:59.962298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8951d37-0458-4cfe-86ef-ef5ea5b47a3f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:59.962715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8951d37-0458-4cfe-86ef-ef5ea5b47a3f None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:40:59.964734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 358/704] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:40:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:40:59.975202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81f8e0-7af6-4b3f-a071-9a9db33dc631 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:40:59.975701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81f8e0-7af6-4b3f-a071-9a9db33dc631 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:40:59.976595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81f8e0-7af6-4b3f-a071-9a9db33dc631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.003039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6c81f8e0-7af6-4b3f-a071-9a9db33dc631 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:00.004520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6c81f8e0-7af6-4b3f-a071-9a9db33dc631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:00.074220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:00.078741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6122ac-ce44-4c7d-a0fa-a2ca0428fc7f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:00.098930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 359/705] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 21 15:41:00.131486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5c152-0d62-4002-ba26-29116288179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:00.133207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c152-0d62-4002-ba26-29116288179d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.133648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c152-0d62-4002-ba26-29116288179d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.134115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c152-0d62-4002-ba26-29116288179d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.143211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81f8e0-7af6-4b3f-a071-9a9db33dc631 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:00.205658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:00.231828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5c152-0d62-4002-ba26-29116288179d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:00.233571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5c152-0d62-4002-ba26-29116288179d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:00.259369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81f8e0-7af6-4b3f-a071-9a9db33dc631 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:00.297748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80a3af-7338-4156-8cc4-570850f4f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:00.298590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80a3af-7338-4156-8cc4-570850f4f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.298875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80a3af-7338-4156-8cc4-570850f4f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.299153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80a3af-7338-4156-8cc4-570850f4f9d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.312633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80a3af-7338-4156-8cc4-570850f4f9d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:00.314155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80a3af-7338-4156-8cc4-570850f4f9d6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-8b80a3af-7338-4156-8cc4-570850f4f9d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:00.332712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:00.333662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:00.334664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:00.334664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:00.334664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:00.334664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:00.334664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 360/706] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:00.335682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0073257-8b11-4157-ae0a-ae43034e60e8 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.335974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0073257-8b11-4157-ae0a-ae43034e60e8 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.336257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0073257-8b11-4157-ae0a-ae43034e60e8 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.337674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 361/707] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:00.338627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d15519-8c55-4276-9202-40e712543c53 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.338919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d15519-8c55-4276-9202-40e712543c53 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.339188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d15519-8c55-4276-9202-40e712543c53 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.341918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 362/708] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:00.342725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d15519-8c55-4276-9202-40e712543c53 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:00.366447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 347/709] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:41:00.366447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c1d1-0fa1-4337-a094-40ca599d0d9e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.366447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c1d1-0fa1-4337-a094-40ca599d0d9e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.366447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612c1d1-0fa1-4337-a094-40ca599d0d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.377033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:00.380172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e612c1d1-0fa1-4337-a094-40ca599d0d9e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:00.381382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e612c1d1-0fa1-4337-a094-40ca599d0d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:00.453706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 348/710] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:41:00.455386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6192e4b-c2d6-441a-ad43-71b8571fc9ec None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.455794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6192e4b-c2d6-441a-ad43-71b8571fc9ec None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.456073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6192e4b-c2d6-441a-ad43-71b8571fc9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.470494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d15519-8c55-4276-9202-40e712543c53 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:00.471715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:00.478833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c6192e4b-c2d6-441a-ad43-71b8571fc9ec None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:00.480648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c6192e4b-c2d6-441a-ad43-71b8571fc9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:00.514626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:00.515318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.515604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.515874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.516606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:00.516917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:00.518350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:00.531213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 363/711] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:00.531872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:00.558665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 349/712] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:41:00.559518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6192e4b-c2d6-441a-ad43-71b8571fc9ec None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:00.631310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:00.648938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eda2d9-cec9-4bbf-92c4-f86098e77da3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:00.675480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6192e4b-c2d6-441a-ad43-71b8571fc9ec None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:00.686212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:00.687037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.687317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.687595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.698359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:00.699859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:00.713425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:00.714444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.714865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.715280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] PATH_INFO: `/v3/services/network` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.716074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:00.717593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:00.717790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:00.718772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:00.719734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:00.719734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:00.719734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:00.719734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:00.719734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 364/713] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:00.720092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:00.728280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:00.740214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:00.741440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:00.742527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:00.745429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 350/714] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:00 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:00.746361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:00.830619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e44b61-f07f-4526-8ebb-5bc434d1e48b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:00.858127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cf379-a960-461f-8ca0-b0c0b3c19dcf None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:00.866107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23d41e-8b54-4478-8051-88b94107fc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:00.866877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23d41e-8b54-4478-8051-88b94107fc61 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.867222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23d41e-8b54-4478-8051-88b94107fc61 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.867501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23d41e-8b54-4478-8051-88b94107fc61 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.868207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23d41e-8b54-4478-8051-88b94107fc61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:00.869673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23d41e-8b54-4478-8051-88b94107fc61 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:00.878562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:00.892177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 365/715] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:41:00 2022] POST /identity/v3/users => generated 254 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:00.893066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23d41e-8b54-4478-8051-88b94107fc61 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:00.920828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc065d-5848-43cd-902e-ee7ce087e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:00.922680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc065d-5848-43cd-902e-ee7ce087e183 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:00.923182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc065d-5848-43cd-902e-ee7ce087e183 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:00.923908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc065d-5848-43cd-902e-ee7ce087e183 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:00.924930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc065d-5848-43cd-902e-ee7ce087e183 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:00.927666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc065d-5848-43cd-902e-ee7ce087e183 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:00.958459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 351/716] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:41:00 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:00.958459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc065d-5848-43cd-902e-ee7ce087e183 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:01.100294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23d41e-8b54-4478-8051-88b94107fc61 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:01.112876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:01.157304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:01.158044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:01.158326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:01.158607 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:01.159336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:01.159643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:01.161064 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:01.175862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 366/717] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:01.183637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:01.185497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc065d-5848-43cd-902e-ee7ce087e183 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:01.234725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:01.235866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:01.235866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:01.236020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:01.236665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] RBAC: Adding query filter params (name=network) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:01.236973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:01.238448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:01.250000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 352/718] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:01.253042 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:01.265148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:01.367911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027a134-a03e-458a-89dd-1c48eeaf32e9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:01.372678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f67dc-56a8-4198-9220-10fabc933bb0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:01.401125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:01.426771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:01.427468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:01.427751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:01.428030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:01.428752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] RBAC: Adding query filter params (type=network) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:01.429060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:01.430650 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:01.443290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:01.444125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:01.444411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:01.444690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:01.450067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 353/719] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:01.458648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:01.461123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:01.498190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:01.499140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:01.500478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 367/720] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:01.506337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:01.657993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce6ef8b-a8d1-41a5-86a7-06fd7fcd029d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:01.706037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de3b1d-7472-4abc-8ddd-33eb024995ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:01.706723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de3b1d-7472-4abc-8ddd-33eb024995ab None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:01.707012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de3b1d-7472-4abc-8ddd-33eb024995ab None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:01.707293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de3b1d-7472-4abc-8ddd-33eb024995ab None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:01.708100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de3b1d-7472-4abc-8ddd-33eb024995ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:01.708407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de3b1d-7472-4abc-8ddd-33eb024995ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:01.709896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de3b1d-7472-4abc-8ddd-33eb024995ab None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:01.732350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 354/721] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:01.737915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:01.886641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d35442-7c16-4ed3-b6b9-59341e5a8910 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:01.907066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:01.937479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d385d-375a-46b6-83dc-47cb616e62d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:01.938568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d385d-375a-46b6-83dc-47cb616e62d4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:01.939142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d385d-375a-46b6-83dc-47cb616e62d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:01.939620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d385d-375a-46b6-83dc-47cb616e62d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:01.940673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d385d-375a-46b6-83dc-47cb616e62d4 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=46e5ca80b7834ab7bc951b454c9aae49) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:01.941219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d385d-375a-46b6-83dc-47cb616e62d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:01.942944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d385d-375a-46b6-83dc-47cb616e62d4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:01.950032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 368/722] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:01 2022] GET /identity/v3/role_assignments?user.id=46e5ca80b7834ab7bc951b454c9aae49&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:02.105509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e60c9-6fb5-40ac-9227-ddaa3c2d1783 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:02.105950 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e60c9-6fb5-40ac-9227-ddaa3c2d1783 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:02.106076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e60c9-6fb5-40ac-9227-ddaa3c2d1783 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:02.107869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 355/723] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:02.116737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167929-61f4-4cc0-818d-98c6f2bb8d92 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:02.117297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167929-61f4-4cc0-818d-98c6f2bb8d92 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:02.117537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167929-61f4-4cc0-818d-98c6f2bb8d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:02.147212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b9167929-61f4-4cc0-818d-98c6f2bb8d92 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:02.149011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b9167929-61f4-4cc0-818d-98c6f2bb8d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:02.230781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 369/724] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:41:02.279275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e60c9-6fb5-40ac-9227-ddaa3c2d1783 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:02.346151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:02.412498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e60c9-6fb5-40ac-9227-ddaa3c2d1783 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:02.469486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e74303-8ca7-4e82-9c12-b443e767f5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:02.470340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e74303-8ca7-4e82-9c12-b443e767f5b0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:02.470618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e74303-8ca7-4e82-9c12-b443e767f5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:02.470889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e74303-8ca7-4e82-9c12-b443e767f5b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:02.488440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e74303-8ca7-4e82-9c12-b443e767f5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-67e74303-8ca7-4e82-9c12-b443e767f5b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:02.496502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:02.497483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:02.498359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:02.499618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 356/725] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:02.506944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9167929-61f4-4cc0-818d-98c6f2bb8d92 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:02.706182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9167929-61f4-4cc0-818d-98c6f2bb8d92 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:02.708647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:02.759497 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:02.760104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:02.760392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:02.760670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:02.761434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:02.761742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:02.763156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:02.778134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 370/726] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:02.784406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e74303-8ca7-4e82-9c12-b443e767f5b0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:02.900977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e74303-8ca7-4e82-9c12-b443e767f5b0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:02.914152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:02.951156 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:02.951965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:02.952250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:02.952524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:02.964458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:02.965947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:02.984331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:02.985256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:02.986313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:02.986313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:02.986313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:02.986313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:02.986313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 357/727] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:02.992367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:03.182213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e472fb6-4d32-439d-9cd6-c3db61762e44 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:03.237817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:03.237817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:03.237817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:03.237817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:03.238234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:03.238234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:03.240101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:03.263781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 371/728] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:03.274963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:03.445996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9cd20-9f18-4044-8831-bf5e8765afa3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:03.481017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:03.481880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:03.482167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:03.482448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:03.499004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:03.500480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:03.515615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:03.516649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:03.517902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 358/729] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:03.525389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:03.560399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:03.654318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c42e6a-221e-4e20-b88d-7086b8a33954 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:03.692620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a35d3-6508-42e1-b4d7-73f8de2f8499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:03.693311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a35d3-6508-42e1-b4d7-73f8de2f8499 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:03.693595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a35d3-6508-42e1-b4d7-73f8de2f8499 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:03.693894 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a35d3-6508-42e1-b4d7-73f8de2f8499 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:03.694830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a35d3-6508-42e1-b4d7-73f8de2f8499 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:03.695142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a35d3-6508-42e1-b4d7-73f8de2f8499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:03.696706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a35d3-6508-42e1-b4d7-73f8de2f8499 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:03.722639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 372/730] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jun 21 15:41:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:03.729079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:03.750026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d448eb2-11a7-4337-9ecf-f255a58e28f7 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:03.750664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d448eb2-11a7-4337-9ecf-f255a58e28f7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:03.751145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d448eb2-11a7-4337-9ecf-f255a58e28f7 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:03.753112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 373/731] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:41:03.754881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf303c0a-b1fc-4421-a56a-021ab9fe8f6b None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:03.755703 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf303c0a-b1fc-4421-a56a-021ab9fe8f6b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:03.756184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf303c0a-b1fc-4421-a56a-021ab9fe8f6b None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:03.758566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 374/732] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:03.760317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09176b05-e41a-4b4b-8465-c254d50df6a1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:03.760809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09176b05-e41a-4b4b-8465-c254d50df6a1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:03.761304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09176b05-e41a-4b4b-8465-c254d50df6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:03.769829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:03.782883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-09176b05-e41a-4b4b-8465-c254d50df6a1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:03.785283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-09176b05-e41a-4b4b-8465-c254d50df6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:03.855803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f305f-45d1-4928-87a9-97182fc15f92 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:03.866181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 375/733] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:41:03.867740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1bbd9-ba06-45d2-b2d9-228226d98ac1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:03.868324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1bbd9-ba06-45d2-b2d9-228226d98ac1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:03.868504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1bbd9-ba06-45d2-b2d9-228226d98ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:03.893323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b869b3b-26c1-4eda-86cd-e8facda42916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:03.894417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869b3b-26c1-4eda-86cd-e8facda42916 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:03.895469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869b3b-26c1-4eda-86cd-e8facda42916 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:03.896017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869b3b-26c1-4eda-86cd-e8facda42916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:03.902151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-cab1bbd9-ba06-45d2-b2d9-228226d98ac1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:03.903571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cab1bbd9-ba06-45d2-b2d9-228226d98ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:03.966286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:03.971686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b869b3b-26c1-4eda-86cd-e8facda42916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:03.973412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b869b3b-26c1-4eda-86cd-e8facda42916 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:03.982195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 376/734] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:41:03.983395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1bbd9-ba06-45d2-b2d9-228226d98ac1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:03.997588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:04.089100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 359/735] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 21 15:41:04.090502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962da3a8-9a43-4926-bb54-2a605417f1ae None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.090795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962da3a8-9a43-4926-bb54-2a605417f1ae None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.091076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962da3a8-9a43-4926-bb54-2a605417f1ae None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.092532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 360/736] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:04.093039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962da3a8-9a43-4926-bb54-2a605417f1ae None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.106165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1bbd9-ba06-45d2-b2d9-228226d98ac1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.143782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a037620-25be-4003-8980-79359003d3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.144624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a037620-25be-4003-8980-79359003d3fe None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.144907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a037620-25be-4003-8980-79359003d3fe None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.145539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a037620-25be-4003-8980-79359003d3fe None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.146244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a037620-25be-4003-8980-79359003d3fe None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:04.147638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a037620-25be-4003-8980-79359003d3fe None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-0a037620-25be-4003-8980-79359003d3fe None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:04.162383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:04.163363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:04.164389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 377/737] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Jun 21 15:41:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:04.165235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a037620-25be-4003-8980-79359003d3fe None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.177666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:04.208307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962da3a8-9a43-4926-bb54-2a605417f1ae None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.235052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:04.258250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66255f-b396-4960-945f-acba9c3dbe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.259683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66255f-b396-4960-945f-acba9c3dbe39 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.260090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66255f-b396-4960-945f-acba9c3dbe39 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.260368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66255f-b396-4960-945f-acba9c3dbe39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.275406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66255f-b396-4960-945f-acba9c3dbe39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:04.277560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a037620-25be-4003-8980-79359003d3fe None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-fc66255f-b396-4960-945f-acba9c3dbe39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:04.285032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:04.286062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 361/738] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:41:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:04.292475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d3426-884c-4b24-a820-479cf552e0de None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.293280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d3426-884c-4b24-a820-479cf552e0de None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.293280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d3426-884c-4b24-a820-479cf552e0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.311688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b13d3426-884c-4b24-a820-479cf552e0de None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:04.313032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b13d3426-884c-4b24-a820-479cf552e0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:04.313570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.314423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.314710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.314982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.327457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:04.328954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:04.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:04.348346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:04.349870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:04.349870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:04.349870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:04.349870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:04.349870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 378/739] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:04.350683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.395405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 362/740] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 21 15:41:04.396363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d3426-884c-4b24-a820-479cf552e0de None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.423604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:04.476809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3acde-c21c-4f27-8c8d-e41027e88d98 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.506633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d3426-884c-4b24-a820-479cf552e0de None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.515034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.515718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.516004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.516279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.516995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:04.517302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:04.518204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:04.518735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:04.523669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 379/741] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jun 21 15:41:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:04.524454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.543452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.544135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.544413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.544686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.545440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] RBAC: Adding query filter params (name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:04.545745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:04.547520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:04.556396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 363/742] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:41:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:04.557503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.631115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e9eb-0058-433e-9abe-d16235133ed9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.659573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:04.662428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978b51-fafd-4099-a3e6-86817288b2ee None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.672235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985b93-e455-4752-9093-2910577718d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.672905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985b93-e455-4752-9093-2910577718d2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.673186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985b93-e455-4752-9093-2910577718d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.673444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985b93-e455-4752-9093-2910577718d2 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.674213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985b93-e455-4752-9093-2910577718d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:04.674517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985b93-e455-4752-9093-2910577718d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:04.675898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985b93-e455-4752-9093-2910577718d2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:04.689346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 380/743] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:04.690811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bce22-c0dd-42c9-9f8f-578345e83570 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.691101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bce22-c0dd-42c9-9f8f-578345e83570 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.691371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bce22-c0dd-42c9-9f8f-578345e83570 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.692790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 381/744] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:04.693354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bce22-c0dd-42c9-9f8f-578345e83570 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.704635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ca0da-62d7-4742-9f17-6eb364024b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.705314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ca0da-62d7-4742-9f17-6eb364024b46 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.705595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ca0da-62d7-4742-9f17-6eb364024b46 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.705907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ca0da-62d7-4742-9f17-6eb364024b46 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.707202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ca0da-62d7-4742-9f17-6eb364024b46 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:04.708864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ca0da-62d7-4742-9f17-6eb364024b46 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:04.724879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 364/745] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:41:04 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:04.725742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ca0da-62d7-4742-9f17-6eb364024b46 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.786755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:04.856284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bce22-c0dd-42c9-9f8f-578345e83570 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.879510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ca0da-62d7-4742-9f17-6eb364024b46 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:04.915547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.916281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.916619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.916902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.917808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:04.918298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:04.919861 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:04.928455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 382/746] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jun 21 15:41:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:04.929828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:04.932697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b0f4e-e91b-4efd-a957-8f75c4d14b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:04.933953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b0f4e-e91b-4efd-a957-8f75c4d14b2d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.934347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b0f4e-e91b-4efd-a957-8f75c4d14b2d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.935155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b0f4e-e91b-4efd-a957-8f75c4d14b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.949757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b0f4e-e91b-4efd-a957-8f75c4d14b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:04.951283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b0f4e-e91b-4efd-a957-8f75c4d14b2d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-e15b0f4e-e91b-4efd-a957-8f75c4d14b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:04.970265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:04.971204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:04.972071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:04.972071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:04.972071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:04.972071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:04.973832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 365/747] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:04.975227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc2d40-830c-42ac-8265-401da110ea5b None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:04.975672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc2d40-830c-42ac-8265-401da110ea5b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:04.976106 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc2d40-830c-42ac-8265-401da110ea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:04.989105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:04.994977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-55bc2d40-830c-42ac-8265-401da110ea5b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:04.996191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-55bc2d40-830c-42ac-8265-401da110ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:05.056063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a77199-694f-456b-aa05-bad5962f5de3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:05.078215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 366/748] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:05.078987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc2d40-830c-42ac-8265-401da110ea5b None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.101575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:05.113183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:05.113965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:05.114248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:05.114517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:05.123851 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:05.125347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:05.142507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:05.143783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 383/749] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:05.144743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.224452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc2d40-830c-42ac-8265-401da110ea5b None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:05.290093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:05.290779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:05.291054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:05.292240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:05.292240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] RBAC: Adding query filter params (interface=public, service_id=71b15eba8f264f0ead70c50bf25adef5, region_id=RegionOne) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:05.292528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:05.292648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb9b0d-6c02-4a4c-acdc-1c8f58771a1c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:05.293816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:05.297275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:05.308265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 367/750] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/endpoints?service_id=71b15eba8f264f0ead70c50bf25adef5&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:05.308265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.345251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:05.346885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:05.347165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:05.347444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:05.348186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:05.349609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:05.349961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:05.373602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 384/751] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:05.379162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.407493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:05.450995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef41da9-3259-47b0-abba-6f7c55bdb66d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:05.512593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19795f-e70b-4ff5-9a95-133a1f4fc9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:05.513938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19795f-e70b-4ff5-9a95-133a1f4fc9b8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:05.514376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19795f-e70b-4ff5-9a95-133a1f4fc9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:05.514691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af0791-155c-4162-9916-e92f43f1ee33 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:05.514851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19795f-e70b-4ff5-9a95-133a1f4fc9b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:05.532593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19795f-e70b-4ff5-9a95-133a1f4fc9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-ae19795f-e70b-4ff5-9a95-133a1f4fc9b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:05.540901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:05.550516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 368/752] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:05.551856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19795f-e70b-4ff5-9a95-133a1f4fc9b8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.579834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:05.580993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:05.581225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:05.581678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:05.583103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:05.583595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:05.586621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:05.616003 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 385/753] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:05.616405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.650113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:05.704685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19795f-e70b-4ff5-9a95-133a1f4fc9b8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:05.748606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad13333-c98b-4fb9-9ec7-ac6d3c295814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:05.749394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad13333-c98b-4fb9-9ec7-ac6d3c295814 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:05.749858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad13333-c98b-4fb9-9ec7-ac6d3c295814 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:05.750121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad13333-c98b-4fb9-9ec7-ac6d3c295814 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:05.750861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad13333-c98b-4fb9-9ec7-ac6d3c295814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:05.752364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad13333-c98b-4fb9-9ec7-ac6d3c295814 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:05.755814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582bf50a-aee6-4bb0-b3d4-830307663fc6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:05.760519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 369/754] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:05.761713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad13333-c98b-4fb9-9ec7-ac6d3c295814 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.765233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:05.812209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:05.814477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:05.815005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:05.815489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:05.827388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:05.830191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:05.852215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:05.853249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:05.854252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:05.855771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 386/755] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:05.858417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.893035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad13333-c98b-4fb9-9ec7-ac6d3c295814 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:05.934016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:05.934694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:05.935030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:05.935306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:05.936106 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:05.936413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:05.938520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:05.948374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 370/756] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:05.949164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:05.973534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:06.002001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc77a86-bccc-467d-8d93-88dbf0a6c5e9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:06.037815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:06.056542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:06.057391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:06.057681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:06.057989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:06.059477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=7230dc79eb9844f1aa618d5e1c883b8a) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:06.059793 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:06.061855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:06.073810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 387/757] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/role_assignments?user.id=7230dc79eb9844f1aa618d5e1c883b8a&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:06.073810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:06.103787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7016b7-1ba3-464f-b13b-8114712c79b0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:06.187394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:06.188539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:06.189021 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:06.189549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:06.190738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:06.191301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:06.193393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:06.218855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdcca48-71e8-491b-add9-72488c923884 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:06.254525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 371/758] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jun 21 15:41:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:06.257800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:06.322595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:06.327197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:06.327197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:06.327197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:06.353908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:06.355784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:06.386188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:06.390231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:06.391204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:06.392573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:06.392573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:06.392573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:06.392573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:06.392573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 388/759] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:06.402934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:06.416914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538888f2-beb6-461e-95e7-5b37d7caea4d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:06.441686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:06.464622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cf46c-fa40-4225-b4d4-31bf3643dab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:06.465335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf46c-fa40-4225-b4d4-31bf3643dab9 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:06.465621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf46c-fa40-4225-b4d4-31bf3643dab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:06.465934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cf46c-fa40-4225-b4d4-31bf3643dab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:06.612207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cf46c-fa40-4225-b4d4-31bf3643dab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:06.614090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7deb-4c9d-4f81-9e2f-6d0581cfa42f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:06.617010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cf46c-fa40-4225-b4d4-31bf3643dab9 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:06.691418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:06.692332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:06.692759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:06.693039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:06.693803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:06.694362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:06.696113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:06.710555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 389/760] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:06.715858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:06.809534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:06.852074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 372/761] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:06.853678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993faa6-eefc-4574-8d35-4cfa1f0e9897 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:06.858603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cf46c-fa40-4225-b4d4-31bf3643dab9 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:06.894584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:06.901593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:06.902536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:06.903046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:06.903328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] PATH_INFO: `/v3/users/nova` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:06.911919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:06.913914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:06.929319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:06.930349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:06.931658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 390/762] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:41:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:06.938264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.027528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cf46c-fa40-4225-b4d4-31bf3643dab9 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.065389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6f9b3-ccef-4a2f-a2d3-18000f9e58da None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.070942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4db48-9a66-4509-9651-e2b8696ea55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.072034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4db48-9a66-4509-9651-e2b8696ea55f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.072548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4db48-9a66-4509-9651-e2b8696ea55f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.073037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4db48-9a66-4509-9651-e2b8696ea55f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.088419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4db48-9a66-4509-9651-e2b8696ea55f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.089918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4db48-9a66-4509-9651-e2b8696ea55f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.092418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:07.107572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d2c03-51a0-4c8b-872b-b3196530d60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.108588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d2c03-51a0-4c8b-872b-b3196530d60a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.109073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d2c03-51a0-4c8b-872b-b3196530d60a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.109551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d2c03-51a0-4c8b-872b-b3196530d60a None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.110675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d2c03-51a0-4c8b-872b-b3196530d60a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-03a4db48-9a66-4509-9651-e2b8696ea55f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:07.111012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:07.111861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:07.113168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:07.113168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:07.113168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:07.113168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:07.113168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 373/763] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:07.113924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1290fa0-dd56-4c9f-a3b2-cf4cfd60287c None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.114173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d2c03-51a0-4c8b-872b-b3196530d60a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.114272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1290fa0-dd56-4c9f-a3b2-cf4cfd60287c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.114471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1290fa0-dd56-4c9f-a3b2-cf4cfd60287c None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.116016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 374/764] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:07.116375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d2c03-51a0-4c8b-872b-b3196530d60a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.117656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1290fa0-dd56-4c9f-a3b2-cf4cfd60287c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.136569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 391/765] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:41:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:07.138196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10718d0-10e8-4b3d-860a-a8f2c2184bf1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.138491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10718d0-10e8-4b3d-860a-a8f2c2184bf1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.138772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10718d0-10e8-4b3d-860a-a8f2c2184bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.157469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f10718d0-10e8-4b3d-860a-a8f2c2184bf1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:07.158746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f10718d0-10e8-4b3d-860a-a8f2c2184bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:07.221684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:07.239737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 392/766] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:07.240693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10718d0-10e8-4b3d-860a-a8f2c2184bf1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.253795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1290fa0-dd56-4c9f-a3b2-cf4cfd60287c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.295906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.296588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.296875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.297154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.297915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:07.298226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.299612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.314147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 375/767] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:07.315759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.373889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10718d0-10e8-4b3d-860a-a8f2c2184bf1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.384553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:07.413025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.413805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.414113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.414410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.425863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.426735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a50db9-d227-432f-b589-ed7996f6ba2f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.427354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:07.448401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:07.449376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:07.450300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:07.450300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:07.450300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:07.450300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:07.450636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 393/768] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:07.451711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.466748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.468015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.468509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.469059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] PATH_INFO: `/v3/services/network` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.469956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.471613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.473801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:07.485679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:07.486728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:07.487544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:07.490626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 376/769] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:07.491818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.567223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee49ed-9d89-4135-a94d-581b9dac931c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.602706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7314a-941d-4206-ba48-28e6f8e5d719 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.605181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.606505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.606694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.607084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.614713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.616566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:07.632287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:07.633232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:07.634156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:07.634156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:07.634156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:07.634156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:07.634156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:07.634156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:07.634156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:07.634156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:07.638336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 394/770] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:07.639617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.644232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.645921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.646450 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.646949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.647955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:07.648565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.650303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.663928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:07.665485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 377/771] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:07.666645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.726390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:07.750988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7153782-a259-4ed3-9c1b-fe811a20de0a None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.771233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb012b5-5ab6-4fbb-a3cd-2062cee0c7de None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.787368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.788054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.788401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.788686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.789446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] RBAC: Adding query filter params (name=network) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:07.789737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.791276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.797000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 395/772] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:07.797990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.810711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.811752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.812261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.812783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.813835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:07.814345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.815977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.828761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 378/773] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:07.829893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.904558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:07.913680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49247c9-f21d-4e4f-9421-d26f51149efd None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.936853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9dce0-10fa-4137-87d4-e64757e6ca54 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:07.951307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.952134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.952422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.952729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.956646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:07.962883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.964639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.975315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:07.976478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:07.976997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:07.977496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:07.978471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] RBAC: Adding query filter params (type=network) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:07.979010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:07.980170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:07.981021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:07.981928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 396/774] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:07.982838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:07.983598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:07.989905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 379/775] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:07.991054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:08.091467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7eca-c17f-4775-9dd1-3d9790132482 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:08.110903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41a8f9-0eb0-4fb8-98af-927e21760424 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:08.132938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:08.133626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.133944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.134220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.135073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=fd307c87c08b42db9484dbb1c0d223e1) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:08.135389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:08.136860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:08.149810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 397/776] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/role_assignments?user.id=fd307c87c08b42db9484dbb1c0d223e1&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:08.149810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:08.153434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cda29-7e06-4f9c-9d46-9cc54dcba70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:08.154236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cda29-7e06-4f9c-9d46-9cc54dcba70b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.154577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cda29-7e06-4f9c-9d46-9cc54dcba70b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.154921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cda29-7e06-4f9c-9d46-9cc54dcba70b None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.155807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cda29-7e06-4f9c-9d46-9cc54dcba70b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:08.156180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cda29-7e06-4f9c-9d46-9cc54dcba70b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:08.157802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cda29-7e06-4f9c-9d46-9cc54dcba70b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:08.164137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:08.178307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 380/777] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:08.180065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6e81-79fd-45eb-8e3c-db48a40ac563 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.180424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6e81-79fd-45eb-8e3c-db48a40ac563 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.180774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f6e81-79fd-45eb-8e3c-db48a40ac563 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.187078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 381/778] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:08.188113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f6e81-79fd-45eb-8e3c-db48a40ac563 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:08.263413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:08.284794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99dc6a-870a-4328-a30c-8d883c8d9308 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:08.327634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a2d2c-b92d-4c04-abe3-b8652a148ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:08.328316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2d2c-b92d-4c04-abe3-b8652a148ebc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.328601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2d2c-b92d-4c04-abe3-b8652a148ebc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.328878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a2d2c-b92d-4c04-abe3-b8652a148ebc None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.329683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a2d2c-b92d-4c04-abe3-b8652a148ebc None admin] RBAC: Adding query filter params (interface=public, service_id=928befecb20e4cee931c634da28664de, region_id=RegionOne) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:08.330023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a2d2c-b92d-4c04-abe3-b8652a148ebc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:08.331419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a2d2c-b92d-4c04-abe3-b8652a148ebc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:08.342123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 398/779] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jun 21 15:41:08 2022] GET /identity/v3/endpoints?service_id=928befecb20e4cee931c634da28664de&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:08.347765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f6e81-79fd-45eb-8e3c-db48a40ac563 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:08.348449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10af35c-2b7f-4e0b-b5eb-f08505d6aa69 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.348951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10af35c-2b7f-4e0b-b5eb-f08505d6aa69 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.349426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10af35c-2b7f-4e0b-b5eb-f08505d6aa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.368017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d10af35c-2b7f-4e0b-b5eb-f08505d6aa69 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:08.369654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d10af35c-2b7f-4e0b-b5eb-f08505d6aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:08.391626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ffc9a-0f6a-4500-bb74-d409c6507ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:08.392535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ffc9a-0f6a-4500-bb74-d409c6507ad0 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.392820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ffc9a-0f6a-4500-bb74-d409c6507ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.393101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ffc9a-0f6a-4500-bb74-d409c6507ad0 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/46e5ca80b7834ab7bc951b454c9aae49/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.395923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ffc9a-0f6a-4500-bb74-d409c6507ad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=46e5ca80b7834ab7bc951b454c9aae49, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:08.397704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ffc9a-0f6a-4500-bb74-d409c6507ad0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:08.421803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 382/780] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:41:08 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/46e5ca80b7834ab7bc951b454c9aae49/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:41:08.421803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ffc9a-0f6a-4500-bb74-d409c6507ad0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:08.484717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 399/781] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:08.486182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2801a5-9e07-4753-91dd-acf74430f999 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.486473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2801a5-9e07-4753-91dd-acf74430f999 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.487275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2801a5-9e07-4753-91dd-acf74430f999 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.489185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 400/782] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:08.494939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476202-0eaf-4298-920c-e544159ac4f1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.495502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476202-0eaf-4298-920c-e544159ac4f1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.495785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35476202-0eaf-4298-920c-e544159ac4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.502902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:08.518705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-35476202-0eaf-4298-920c-e544159ac4f1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:08.520972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-35476202-0eaf-4298-920c-e544159ac4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:08.555664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ffc9a-0f6a-4500-bb74-d409c6507ad0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:08.558189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:08.607520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25293d-eb47-4306-85c3-d02887373643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:08.608346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25293d-eb47-4306-85c3-d02887373643 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.608631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25293d-eb47-4306-85c3-d02887373643 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.609060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25293d-eb47-4306-85c3-d02887373643 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.610190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25293d-eb47-4306-85c3-d02887373643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:08.611879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25293d-eb47-4306-85c3-d02887373643 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:08.617900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 401/783] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:08.618981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35476202-0eaf-4298-920c-e544159ac4f1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:08.624571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 383/784] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:41:08 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:08.639597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25293d-eb47-4306-85c3-d02887373643 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:08.763137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35476202-0eaf-4298-920c-e544159ac4f1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:08.792966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25293d-eb47-4306-85c3-d02887373643 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:08.806888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:08.814706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:08.816131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.816423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.816702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.817670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:08.820061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:08.834440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:08.835472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:08.836299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 402/785] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Jun 21 15:41:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:08.841788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:08.858568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3b947-8c5d-47c5-b8f4-bc45c719f57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:08.859466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3b947-8c5d-47c5-b8f4-bc45c719f57a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:08.859745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3b947-8c5d-47c5-b8f4-bc45c719f57a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:08.860017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3b947-8c5d-47c5-b8f4-bc45c719f57a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:08.877435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3b947-8c5d-47c5-b8f4-bc45c719f57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-88a3b947-8c5d-47c5-b8f4-bc45c719f57a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:08.891130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:08.892107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:08.892947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:08.898262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 384/786] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:08.905270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3b947-8c5d-47c5-b8f4-bc45c719f57a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:08.943957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:08.979779 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92dac50-fdd1-489a-bd23-391e840b247c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:09.017640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:09.018712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:09.019203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:09.020044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:09.020958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:09.021492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:09.023491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:09.029219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 403/787] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jun 21 15:41:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:09.034418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:09.083320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3b947-8c5d-47c5-b8f4-bc45c719f57a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:09.126497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:09.129801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:09.130296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:09.130812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:09.135105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:09.135651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:09.137313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:09.157612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 385/788] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:09.163247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:09.185038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:09.293504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4d317-27ef-4ee6-91e7-61a06bfd9c62 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:09.296724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:09.321192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d05c80-27d1-44ea-90e9-42549fb994ff None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:09.349954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:09.350754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:09.351168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:09.351542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:09.352272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:09.352592 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:09.354031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:09.361031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 404/789] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Jun 21 15:41:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:09.366692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:09.399468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:09.400293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:09.400600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:09.400844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:09.423075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:09.424578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:09.443285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:09.444262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:09.445324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:09.445324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:09.445324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:09.445324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:09.445324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 386/790] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:09.450389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:09.557738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9451ab-39d2-4d7b-9efc-1f124cb7d655 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:09.593533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a809609-1709-4556-ae77-adf2534c5f98 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:09.607337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:09.631221 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177b36f-ec36-4823-985b-307f41fbff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:09.632981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b36f-ec36-4823-985b-307f41fbff23 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:09.633398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b36f-ec36-4823-985b-307f41fbff23 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:09.633677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b36f-ec36-4823-985b-307f41fbff23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:09.634465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177b36f-ec36-4823-985b-307f41fbff23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:09.635940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177b36f-ec36-4823-985b-307f41fbff23 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:09.648813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:09.649548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:09.649864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:09.650252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:09.651096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:09.651419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:09.652849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:09.665886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:09.671708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 387/791] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:09.677812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:09.684897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 405/792] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:41:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:41:09.816512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907775e-8b77-4f82-a5ec-083ba93040c4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:09.883491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:09.884740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:09.885218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:09.885682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:09.899436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:09.901617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:09.926577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:09.927605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:41:09.928512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:09.931542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 388/793] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:09.938126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177b36f-ec36-4823-985b-307f41fbff23 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:10.055604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177b36f-ec36-4823-985b-307f41fbff23 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:10.098616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:10.099283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:10.099575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:10.099849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:10.100747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:10.101056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:10.102559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:10.124351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 406/794] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jun 21 15:41:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:10.127532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:10.203172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:10.249896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b5bb12-ffe7-4714-b475-197d412c6c4e None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:10.292430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd1de1-7233-4b31-8232-6114c5737a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:10.293247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1de1-7233-4b31-8232-6114c5737a8e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:10.293531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1de1-7233-4b31-8232-6114c5737a8e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:10.293841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd1de1-7233-4b31-8232-6114c5737a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:10.309089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd1de1-7233-4b31-8232-6114c5737a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:10.310613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd1de1-7233-4b31-8232-6114c5737a8e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-15cd1de1-7233-4b31-8232-6114c5737a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:10.332540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:10.333470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:10.334814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:10.334814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:10.334814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:10.334814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:10.334814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 389/795] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:10.340165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:10.399378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:10.461940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc06710-9cdc-4c86-b52e-53f5c822976d None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:10.462267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc06710-9cdc-4c86-b52e-53f5c822976d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:10.462507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc06710-9cdc-4c86-b52e-53f5c822976d None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:10.464140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 390/796] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:10.465510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6ede3-d73d-4ba7-bf81-7d6fcea81bcc None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:10.469584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf642f0-2fb7-49cb-8f3a-dd53a63408b4 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:10.469907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf642f0-2fb7-49cb-8f3a-dd53a63408b4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:10.470189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf642f0-2fb7-49cb-8f3a-dd53a63408b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:10.506870 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7bf642f0-2fb7-49cb-8f3a-dd53a63408b4 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:10.507955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:10.508904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:10.509410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:10.509948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:10.510893 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:10.511400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:10.512882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7bf642f0-2fb7-49cb-8f3a-dd53a63408b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:10.514520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:10.544704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 407/797] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:10.548204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:10.662124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 391/798] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:10.708079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf642f0-2fb7-49cb-8f3a-dd53a63408b4 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:10.736938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:10.743156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd19a5-5768-4c4c-bf45-44fddd40f74d None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:10.764641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:10.820351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:10.821230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:10.821512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:10.821799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:10.842848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:10.844440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:10.861590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf642f0-2fb7-49cb-8f3a-dd53a63408b4 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:10.865211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:10.866139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:10.867498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:10.867498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:10.867498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:10.867498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:10.867498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:10.867498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:10.867498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:10.867498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:10.873842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 408/799] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:10.879715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:10.900344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974d869-934a-4eff-901a-e8673e21d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:10.901164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974d869-934a-4eff-901a-e8673e21d027 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:10.901448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974d869-934a-4eff-901a-e8673e21d027 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:10.901731 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974d869-934a-4eff-901a-e8673e21d027 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:10.914107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974d869-934a-4eff-901a-e8673e21d027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-6974d869-934a-4eff-901a-e8673e21d027 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:10.921985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:10.922948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:10.924013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 392/800] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:41:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:10.933295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974d869-934a-4eff-901a-e8673e21d027 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:11.088918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f113f-065f-466a-8eb0-427e4fda6228 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:11.115457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:11.119826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974d869-934a-4eff-901a-e8673e21d027 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:11.130756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af829143-8f0e-4d28-a957-6c8eaf641837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:11.132199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af829143-8f0e-4d28-a957-6c8eaf641837 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.132312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af829143-8f0e-4d28-a957-6c8eaf641837 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.132587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af829143-8f0e-4d28-a957-6c8eaf641837 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.138108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af829143-8f0e-4d28-a957-6c8eaf641837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:11.138877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af829143-8f0e-4d28-a957-6c8eaf641837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:11.140402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af829143-8f0e-4d28-a957-6c8eaf641837 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:11.153565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 409/801] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:11.159636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f2aba-dde1-44f5-a742-587d84f0ff01 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.159928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f2aba-dde1-44f5-a742-587d84f0ff01 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.160208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f2aba-dde1-44f5-a742-587d84f0ff01 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.161564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 410/802] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:11.162708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2333a7f-9ff5-4e2b-bfa4-264254b9227d None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.163164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2333a7f-9ff5-4e2b-bfa4-264254b9227d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.163436 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2333a7f-9ff5-4e2b-bfa4-264254b9227d None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.169896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 411/803] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:11.169896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2333a7f-9ff5-4e2b-bfa4-264254b9227d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:11.171875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6e4e-6d13-42c1-b841-772feb9f16e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:11.172762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6e4e-6d13-42c1-b841-772feb9f16e6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.173207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6e4e-6d13-42c1-b841-772feb9f16e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.173614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6e4e-6d13-42c1-b841-772feb9f16e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.174540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6e4e-6d13-42c1-b841-772feb9f16e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:11.175011 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6e4e-6d13-42c1-b841-772feb9f16e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:11.176539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6e4e-6d13-42c1-b841-772feb9f16e6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:11.186033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 393/804] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:41:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:11.188295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a51b3-dd2b-404f-9245-ebc257ea2829 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.188813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a51b3-dd2b-404f-9245-ebc257ea2829 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.189302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a51b3-dd2b-404f-9245-ebc257ea2829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.195169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:11.211780 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ef5a51b3-dd2b-404f-9245-ebc257ea2829 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:11.213549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ef5a51b3-dd2b-404f-9245-ebc257ea2829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:11.299533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2333a7f-9ff5-4e2b-bfa4-264254b9227d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:11.307099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 394/805] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:11.308783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513888c-31b6-405f-94f7-4aa24ea4191e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.308783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513888c-31b6-405f-94f7-4aa24ea4191e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.308953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513888c-31b6-405f-94f7-4aa24ea4191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.338221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c513888c-31b6-405f-94f7-4aa24ea4191e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:11.339554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c513888c-31b6-405f-94f7-4aa24ea4191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:11.350524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fc4a-e32a-4e3a-9453-dd647f731639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:11.351811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fc4a-e32a-4e3a-9453-dd647f731639 None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.352101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fc4a-e32a-4e3a-9453-dd647f731639 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.352558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605fc4a-e32a-4e3a-9453-dd647f731639 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/7230dc79eb9844f1aa618d5e1c883b8a/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.354595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605fc4a-e32a-4e3a-9453-dd647f731639 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=7230dc79eb9844f1aa618d5e1c883b8a, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:11.356076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605fc4a-e32a-4e3a-9453-dd647f731639 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:11.376217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 412/806] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:41:11 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/7230dc79eb9844f1aa618d5e1c883b8a/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:41:11.376217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fc4a-e32a-4e3a-9453-dd647f731639 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:11.429260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 395/807] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:11.430426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513888c-31b6-405f-94f7-4aa24ea4191e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:11.467765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:11.506281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:11.541580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605fc4a-e32a-4e3a-9453-dd647f731639 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:11.579924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513888c-31b6-405f-94f7-4aa24ea4191e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:11.587325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:11.588532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.589023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.589494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.608144 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:11.610506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:11.622798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd2ff-5773-431e-9683-b558270b040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:11.623758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd2ff-5773-431e-9683-b558270b040d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.624224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd2ff-5773-431e-9683-b558270b040d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.624796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdd2ff-5773-431e-9683-b558270b040d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:11.631207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:11.632327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:11.633338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:11.633338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:11.633338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:11.633338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:11.635549 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 413/808] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:11.636782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:11.649828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdd2ff-5773-431e-9683-b558270b040d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-47fdd2ff-5773-431e-9683-b558270b040d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:11.659416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:11.660377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:11.661213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:11.662068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 396/809] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:11.666722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd2ff-5773-431e-9683-b558270b040d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:11.770501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80066c-ef15-46c7-99d2-aa809d685fda None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:11.775002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:11.800020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdd2ff-5773-431e-9683-b558270b040d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:11.816074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:11.825036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:11.826235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.826949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.827981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] PATH_INFO: `/v3/services/network` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.829590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:11.833072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:11.839991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:11.840795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:11.841179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:11.841556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:11.842439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:11.842852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:11.844344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:11.849365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:11.850637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 21 15:41:11.851734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:11.854476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 414/810] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:11 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:11.856352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:11.872355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 397/811] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:11.874034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:11.978859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903783cd-daeb-482f-93a9-02049d37f0bf None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:12.026208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58e333-2ed4-4adc-af46-4ea5f4d78795 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:12.036035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:12.036947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.037372 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.037870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.039211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:12.039678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:12.041272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:12.057160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 415/812] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:12.058512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:12.121798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:12.171913 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca19995-e9b7-4fb0-b2f2-67ff437838e6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:12.209579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:12.210729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.211150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.211586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.222207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:12.223945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:12.242218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:12.243306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:12.244358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:12.247574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 416/813] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:41:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:12.248545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:12.263365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:12.264626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.265238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.266015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.267451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] RBAC: Adding query filter params (name=network) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:12.268112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:12.271104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:12.280267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 398/814] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:12.281939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:12.289102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:12.377721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb9833-4ed4-4a91-ae2a-2926df582f0d None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:12.395600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:12.415098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923373e5-3020-4461-aad3-4552fb4f2720 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:12.417408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:12.418336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.418684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.418968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.432713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:12.435249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:12.461130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcda2b-4a87-454c-b6e2-d278ef297183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:12.462200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcda2b-4a87-454c-b6e2-d278ef297183 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.462647 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcda2b-4a87-454c-b6e2-d278ef297183 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.463074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcda2b-4a87-454c-b6e2-d278ef297183 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:12.464460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:12.465294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:12.466192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:12.466192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:12.466192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:12.466192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:12.466406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcda2b-4a87-454c-b6e2-d278ef297183 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:12.466406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcda2b-4a87-454c-b6e2-d278ef297183 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:12.471617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 417/815] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:12.471703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbcda2b-4a87-454c-b6e2-d278ef297183 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:12.471784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:12.499289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 399/816] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jun 21 15:41:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:12.502025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5296bf-56b9-493b-9abb-7d68afb2e103 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.502564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5296bf-56b9-493b-9abb-7d68afb2e103 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.503047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5296bf-56b9-493b-9abb-7d68afb2e103 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.505024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 400/817] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:12.506015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5296bf-56b9-493b-9abb-7d68afb2e103 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:12.585470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:12.634369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5296bf-56b9-493b-9abb-7d68afb2e103 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:12.671137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:12.672067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.672500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.672794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.673518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:12.673969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:12.675276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:12.687506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 401/818] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:12.688314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:12.718914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6418a-2ef8-420a-9e8b-776128ec3b4b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:12.752903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d26cc4-f53a-4cde-900b-c07d22fa8ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:12.753581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d26cc4-f53a-4cde-900b-c07d22fa8ca4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.753889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d26cc4-f53a-4cde-900b-c07d22fa8ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.754168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d26cc4-f53a-4cde-900b-c07d22fa8ca4 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.754883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d26cc4-f53a-4cde-900b-c07d22fa8ca4 None admin] RBAC: Adding query filter params (type=network) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:12.755190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d26cc4-f53a-4cde-900b-c07d22fa8ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:12.756734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d26cc4-f53a-4cde-900b-c07d22fa8ca4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:12.766678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 418/819] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:12.766678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d862-5c4a-4e34-9cb2-b7a5c1268e42 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.766678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d862-5c4a-4e34-9cb2-b7a5c1268e42 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.766678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d862-5c4a-4e34-9cb2-b7a5c1268e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.781023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ccd4d862-5c4a-4e34-9cb2-b7a5c1268e42 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:12.782239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ccd4d862-5c4a-4e34-9cb2-b7a5c1268e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:12.802354 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39a87f-ac8f-459f-8a2d-fa0acb9791d1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:12.810200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:12.826172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:12.840323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:12.841114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:12.841388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:12.841699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:12.853244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:12.855046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:12.856954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 419/820] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:12.857886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4d862-5c4a-4e34-9cb2-b7a5c1268e42 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:12.888633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:12.890145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:12.891226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:12.891226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:12.891226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:12.891226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:12.892139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 402/821] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:12.897220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:12.973717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4d862-5c4a-4e34-9cb2-b7a5c1268e42 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.005425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e0821-f9d5-4426-b768-08ba61a6ba80 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.009669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.010515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.010792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.011063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.018665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.020985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:13.035480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:13.036450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:13.037414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 420/822] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:41:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:13.038669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:13.047112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40f152-067d-42a9-b2a0-11e541a0f9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.047833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f152-067d-42a9-b2a0-11e541a0f9af None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.048123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f152-067d-42a9-b2a0-11e541a0f9af None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.048373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40f152-067d-42a9-b2a0-11e541a0f9af None admin] PATH_INFO: `/v3/endpoints` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.049168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40f152-067d-42a9-b2a0-11e541a0f9af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.050612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40f152-067d-42a9-b2a0-11e541a0f9af None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.066485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:13.085802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 403/823] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:41:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:13.085802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40f152-067d-42a9-b2a0-11e541a0f9af None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:13.113373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:13.186834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b259d4-73be-4745-a1b9-6ad592b3ad40 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.205674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40f152-067d-42a9-b2a0-11e541a0f9af None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.246073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.247173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.248726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.248846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.249501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.250983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.263668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.264707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:13.265054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:13.266074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:13.266925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 421/824] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:13.267729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:13.268522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.269014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.270351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:13.270875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.272687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.290127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 404/825] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:13.291715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:13.388492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:13.405112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:13.406092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367c94b-5b42-4ad0-8c3b-a5171ba70496 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.421811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f37271-474e-4484-9841-6916f1fff2c5 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.443262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.444246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.444504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.444918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.445976 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:13.446363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.447907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.459122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.459958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.460237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.460511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.461332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:13.461642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.463046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.468430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 405/826] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:13.468949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 422/827] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:41:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:13.469207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:13.478857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:13.589162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda4a82-3473-49a2-8685-7ac2eddad692 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.593198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dec042-9a0f-4509-b7e5-b9cb0704e467 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.624400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c8b46-7978-489a-bad7-257b457281af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.625073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c8b46-7978-489a-bad7-257b457281af None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.625357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c8b46-7978-489a-bad7-257b457281af None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.625630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c8b46-7978-489a-bad7-257b457281af None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.626377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c8b46-7978-489a-bad7-257b457281af None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:13.626686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c8b46-7978-489a-bad7-257b457281af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.627302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b22cd-9877-4cc8-8444-dbd3362a6622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.628053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c8b46-7978-489a-bad7-257b457281af None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.628246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b22cd-9877-4cc8-8444-dbd3362a6622 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.628525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b22cd-9877-4cc8-8444-dbd3362a6622 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.628794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b22cd-9877-4cc8-8444-dbd3362a6622 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.632792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 423/828] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:13.633492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c8b46-7978-489a-bad7-257b457281af None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:13.636274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b22cd-9877-4cc8-8444-dbd3362a6622 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.638222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b22cd-9877-4cc8-8444-dbd3362a6622 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-d25b22cd-9877-4cc8-8444-dbd3362a6622 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:13.654160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:13.655083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:13.661035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 406/829] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:13.661624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:13.661711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdda8bd-1599-4422-8ffb-58bfc6ce32ed None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.661711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdda8bd-1599-4422-8ffb-58bfc6ce32ed None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.661711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdda8bd-1599-4422-8ffb-58bfc6ce32ed None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.661711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 407/830] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:13.665500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdda8bd-1599-4422-8ffb-58bfc6ce32ed None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:13.683190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:13.769712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c8b46-7978-489a-bad7-257b457281af None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.786260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdda8bd-1599-4422-8ffb-58bfc6ce32ed None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:13.811948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a7689-f0dd-4793-bb91-3e6428935871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.813486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7689-f0dd-4793-bb91-3e6428935871 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.813486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7689-f0dd-4793-bb91-3e6428935871 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.813486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7689-f0dd-4793-bb91-3e6428935871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.828755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50f326-88c6-41f6-a7b6-7d210aaea2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:13.829533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f326-88c6-41f6-a7b6-7d210aaea2bb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.829939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f326-88c6-41f6-a7b6-7d210aaea2bb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.830285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50f326-88c6-41f6-a7b6-7d210aaea2bb None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.831154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50f326-88c6-41f6-a7b6-7d210aaea2bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:13.831532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50f326-88c6-41f6-a7b6-7d210aaea2bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.833046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50f326-88c6-41f6-a7b6-7d210aaea2bb None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.849946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 408/831] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:13.849946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e3829-2637-4332-9cf2-eaa39b29a4f8 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:13.849946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e3829-2637-4332-9cf2-eaa39b29a4f8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:13.849946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e3829-2637-4332-9cf2-eaa39b29a4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:13.873149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-943e3829-2637-4332-9cf2-eaa39b29a4f8 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:13.875507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-943e3829-2637-4332-9cf2-eaa39b29a4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:13.895919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a7689-f0dd-4793-bb91-3e6428935871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:13.898254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a7689-f0dd-4793-bb91-3e6428935871 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:13.910745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:13.948874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:13.954102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 409/832] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:13.955269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e3829-2637-4332-9cf2-eaa39b29a4f8 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.011387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 424/833] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:14.012255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a7689-f0dd-4793-bb91-3e6428935871 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.086199 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e3829-2637-4332-9cf2-eaa39b29a4f8 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.118837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a7689-f0dd-4793-bb91-3e6428935871 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.125946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33db2-67f1-4f19-8a61-2f4e9b72c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.126967 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33db2-67f1-4f19-8a61-2f4e9b72c907 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.127253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33db2-67f1-4f19-8a61-2f4e9b72c907 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.127530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33db2-67f1-4f19-8a61-2f4e9b72c907 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/fd307c87c08b42db9484dbb1c0d223e1/roles/7a558a55cb054e62ac77df614c5bd73d` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.128630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:14.144205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33db2-67f1-4f19-8a61-2f4e9b72c907 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=fd307c87c08b42db9484dbb1c0d223e1, role_id=7a558a55cb054e62ac77df614c5bd73d)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:14.146047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33db2-67f1-4f19-8a61-2f4e9b72c907 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:14.156735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57693c0c-6de8-4196-89cc-88ea57a54906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.157531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693c0c-6de8-4196-89cc-88ea57a54906 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.157843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693c0c-6de8-4196-89cc-88ea57a54906 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.158126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57693c0c-6de8-4196-89cc-88ea57a54906 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.170241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 410/834] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:41:13 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/fd307c87c08b42db9484dbb1c0d223e1/roles/7a558a55cb054e62ac77df614c5bd73d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:41:14.170241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33db2-67f1-4f19-8a61-2f4e9b72c907 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.182482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57693c0c-6de8-4196-89cc-88ea57a54906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-57693c0c-6de8-4196-89cc-88ea57a54906 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:14.192924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:14.193968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:14.195013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 425/835] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:14.201921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57693c0c-6de8-4196-89cc-88ea57a54906 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.235286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:14.304882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33db2-67f1-4f19-8a61-2f4e9b72c907 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.341755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57693c0c-6de8-4196-89cc-88ea57a54906 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.367883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.369388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.369911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.370380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.384862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.386061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.386567 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.387038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.388008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:14.388573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:14.391388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:14.391949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:14.395236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:14.402674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 426/836] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:14.410982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:14.425986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:14.426915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:14.427777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:14.427777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:14.427777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:14.427777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:14.427777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 411/837] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:14.433396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.447622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:14.531486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4634fa-44d1-47d5-a981-a7cec2658aac None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.562306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b9749-c52a-43a7-846e-27c1dda45675 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.564202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:14.566611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.567419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.567699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.567965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.579557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:14.581024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:14.599242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:14.600225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:14.601092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:14.601092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:14.601092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:14.601092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:14.601092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 427/838] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:14.606710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.608479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.609170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.609455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.609991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.611640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:14.611963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:14.613465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:14.629139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 412/839] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:14.635358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.702822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:14.717934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509d0ef-2864-4133-b395-6d134f1966fc None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.755244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5daaf0-f20b-4a34-b1b0-ba60e6e77c21 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.756455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.757278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.757740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.758041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.758874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:14.759181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:14.760601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:14.774696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 428/840] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:14.783405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.799752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.800673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.801167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.801519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.811198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:14.813202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:14.831287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:14.832330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:14.833505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:14.833505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:14.833505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:14.833505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:14.833505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:14.833505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:14.833505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:14.833505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:14.837868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 413/841] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:14.840646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.867587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:14.906541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff7e9c-76a0-4d78-86bb-8ba63308d763 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.944667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:14.945866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:14.946368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:14.946869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:14.955173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:14.956704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:14.957574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b0c4-8fa7-4b18-b397-b61adf73d142 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:14.973016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:14.974265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 21 15:41:14.975255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:14.976361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 429/842] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:14.982079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:14.986300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:15.011557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:15.012596 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:15.013665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:15.014159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:15.015524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:15.016018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:15.018749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:15.039818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 414/843] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:15.040880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:15.100209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab114ac2-7a80-49cf-9de0-9b498874ca5e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:15.134022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:15.136364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:15.137379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:15.137903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:15.138375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:15.139470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:15.139998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:15.141707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:15.156700 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2e69a-0eed-4fdb-8a0c-2fa0917c69fb None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:15.161592 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 430/844] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Jun 21 15:41:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:15.173561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:15.192696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:15.193360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:15.193637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:15.194005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:15.195055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=46e5ca80b7834ab7bc951b454c9aae49) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:15.195405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:15.197146 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:15.210146 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 415/845] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:15 2022] GET /identity/v3/role_assignments?user.id=46e5ca80b7834ab7bc951b454c9aae49&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:15.241221 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:15.288725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1ead-1f80-45ca-9512-b12bdfaa39fe None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:15.327694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c08d4e-6bf2-43c2-8652-f54ccad85e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:15.328372 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08d4e-6bf2-43c2-8652-f54ccad85e53 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:15.328653 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08d4e-6bf2-43c2-8652-f54ccad85e53 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:15.328928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c08d4e-6bf2-43c2-8652-f54ccad85e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:15.407150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c08d4e-6bf2-43c2-8652-f54ccad85e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:15.408655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c08d4e-6bf2-43c2-8652-f54ccad85e53 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:15.470319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:15.514665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 431/846] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:15.520526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:15.627264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab16be5-5b1d-4d22-88a2-8484d07f8e92 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:15.664287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:15.665096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:15.665379 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:15.665655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:15.676764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:15.678382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:15.685741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:15.711039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:15.711986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:15.712884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:15.712884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:15.712884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:15.712884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:15.718093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 416/847] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:15.723186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c08d4e-6bf2-43c2-8652-f54ccad85e53 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:15.873016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c08d4e-6bf2-43c2-8652-f54ccad85e53 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:15.908638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40bf6-f6da-452f-b9c6-4cbbc23c9217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:15.909426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40bf6-f6da-452f-b9c6-4cbbc23c9217 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:15.909644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40bf6-f6da-452f-b9c6-4cbbc23c9217 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:15.909965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40bf6-f6da-452f-b9c6-4cbbc23c9217 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:15.910747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f40bf6-f6da-452f-b9c6-4cbbc23c9217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:15.911060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f40bf6-f6da-452f-b9c6-4cbbc23c9217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:15.912568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f40bf6-f6da-452f-b9c6-4cbbc23c9217 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:15.929943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 432/848] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:15.931697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:16.053685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4446ac0-5e3b-486c-821f-5067ed4388de None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:16.089099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:16.089945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.090256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.090555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.098126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:16.099599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:16.103964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce5c5-340a-4956-9895-301cecc30b21 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.104251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce5c5-340a-4956-9895-301cecc30b21 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.104525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce5c5-340a-4956-9895-301cecc30b21 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.110103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 433/849] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:16.111816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0b8c7-60b4-4a5b-9532-56448ab1115a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.112102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0b8c7-60b4-4a5b-9532-56448ab1115a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.112379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0b8c7-60b4-4a5b-9532-56448ab1115a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:16.118277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:16.119168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:16.120142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:16.120142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:16.120142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:16.120142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:16.120142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:16.120142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:16.120142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:16.120142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:16.121308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 417/850] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:16.129823 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:16.136734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-46f0b8c7-60b4-4a5b-9532-56448ab1115a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:16.138054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-46f0b8c7-60b4-4a5b-9532-56448ab1115a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:16.154168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:16.211267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:16.217926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 434/851] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:16.240583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0b8c7-60b4-4a5b-9532-56448ab1115a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:16.281942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ea2d5-7256-4a3e-ad82-659e3f52fbe8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:16.320601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:16.321866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.322393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.322858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.323840 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:16.324378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:16.326266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:16.345242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 418/852] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:16.346939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:16.367952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0b8c7-60b4-4a5b-9532-56448ab1115a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:16.414437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e6597e-5c9a-4e21-860a-112575828371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:16.415166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6597e-5c9a-4e21-860a-112575828371 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.415448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6597e-5c9a-4e21-860a-112575828371 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.415718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e6597e-5c9a-4e21-860a-112575828371 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.416472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6597e-5c9a-4e21-860a-112575828371 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:16.422837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e6597e-5c9a-4e21-860a-112575828371 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:16.449152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 435/853] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:41:16 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:16.485346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517b3dd-79c8-4574-a8ce-28de9f4d47c6 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.485346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517b3dd-79c8-4574-a8ce-28de9f4d47c6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.486781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517b3dd-79c8-4574-a8ce-28de9f4d47c6 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.487577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 436/854] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:16.494720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e511d-eda4-45f5-8f5f-24d910341331 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.495056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e511d-eda4-45f5-8f5f-24d910341331 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.495293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e511d-eda4-45f5-8f5f-24d910341331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.503366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1164f3-7115-4740-b161-c9aba8d97b61 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:16.518711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-062e511d-eda4-45f5-8f5f-24d910341331 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:16.520242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-062e511d-eda4-45f5-8f5f-24d910341331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:16.526464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:16.527450 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:16.558651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:16.559365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.559652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.559922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.560816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=7230dc79eb9844f1aa618d5e1c883b8a) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:16.561136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:16.562713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:16.570450 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 419/855] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:16 2022] GET /identity/v3/role_assignments?user.id=7230dc79eb9844f1aa618d5e1c883b8a&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:16.631494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 437/856] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:16.659127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:16.780040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c6be-eb3d-4c9b-88df-2c08852b122e None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:16.818791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75107b9f-7826-436a-8673-dd0be2d2a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:16.819892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75107b9f-7826-436a-8673-dd0be2d2a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:16.819892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75107b9f-7826-436a-8673-dd0be2d2a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:16.820250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75107b9f-7826-436a-8673-dd0be2d2a2e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:16.831961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75107b9f-7826-436a-8673-dd0be2d2a2e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-75107b9f-7826-436a-8673-dd0be2d2a2e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:16.840688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:16.841841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:16.842674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 420/857] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:41:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:16.850164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e511d-eda4-45f5-8f5f-24d910341331 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:16.961039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e511d-eda4-45f5-8f5f-24d910341331 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:16.999430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:17.000527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.001057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.001535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.002599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:17.003163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:17.004914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:17.015601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 438/858] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:41:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:17.023019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75107b9f-7826-436a-8673-dd0be2d2a2e2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:17.081593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:17.148123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75107b9f-7826-436a-8673-dd0be2d2a2e2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:17.184679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7e9be-8b7e-488d-9897-c5e9985e263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:17.185556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e9be-8b7e-488d-9897-c5e9985e263a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.185868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e9be-8b7e-488d-9897-c5e9985e263a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.186196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e9be-8b7e-488d-9897-c5e9985e263a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.198693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7e9be-8b7e-488d-9897-c5e9985e263a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:17.200165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7e9be-8b7e-488d-9897-c5e9985e263a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f8d7e9be-8b7e-488d-9897-c5e9985e263a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:17.218891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:17.219802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:17.220675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:17.220675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:17.220675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:17.220675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:17.220675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 421/859] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:17.228077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:17.231412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:17.353086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf0247-ff6f-4206-bea0-8698ffc0d986 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:17.361408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510af6e4-3e47-407c-93ce-ce5a66b30e9a None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.361706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510af6e4-3e47-407c-93ce-ce5a66b30e9a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.361961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510af6e4-3e47-407c-93ce-ce5a66b30e9a None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.363564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 422/860] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:17.368843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79888fa-bb9b-453a-9869-cec78a17899a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.369133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79888fa-bb9b-453a-9869-cec78a17899a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.369367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79888fa-bb9b-453a-9869-cec78a17899a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.388059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b79888fa-bb9b-453a-9869-cec78a17899a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:17.389615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b79888fa-bb9b-453a-9869-cec78a17899a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:17.404981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:17.405687 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.405968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.406245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.406976 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:17.407282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:17.408677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:17.421764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 439/861] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:17.427177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:17.481909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 423/862] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:17.501356 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79888fa-bb9b-453a-9869-cec78a17899a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:17.504289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:17.535991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:17.548165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe26fc1-f789-4cde-8706-44fb0aa290f0 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:17.592997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:17.594078 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.594447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.594795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.605040 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:17.606628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:17.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:17.623430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:17.624348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 440/863] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:41:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:17.631054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:17.648291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79888fa-bb9b-453a-9869-cec78a17899a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:17.696328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:17.697665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.698193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.698660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.699512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:17.701207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:17.715816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:17.716815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:17.717620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:17.720911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 424/864] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Jun 21 15:41:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:17.728569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:17.755553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d8c6c-9187-49c6-b93b-1c29983ae211 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:17.782935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:17.799020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:17.799770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.800052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.800618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.801181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:17.801546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:17.803167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:17.826962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 441/865] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jun 21 15:41:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:17.832519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:17.845906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:17.863837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8a10e-3a65-4ecf-8d9d-34a2a6074adf None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:17.904335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:17.905010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:17.905291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:17.905565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:17.906320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:17.906625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:17.908000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:17.919374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 425/866] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:17.922216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:17.977464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8bf30-e7aa-4196-8a31-c3ccc65a79c1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:18.015775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e47ed-9c1f-4cd3-bcb2-8155b4e0c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:18.016769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e47ed-9c1f-4cd3-bcb2-8155b4e0c603 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:18.017237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e47ed-9c1f-4cd3-bcb2-8155b4e0c603 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:18.017679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e47ed-9c1f-4cd3-bcb2-8155b4e0c603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:18.031504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05616069-4ed9-48ce-841c-164fef0ca8d1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:18.067348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:18.067920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:18.068270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:18.068613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:18.069654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:18.070056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:18.071624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:18.086043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 426/867] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:18.089761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:18.112875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e47ed-9c1f-4cd3-bcb2-8155b4e0c603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:18.114575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e47ed-9c1f-4cd3-bcb2-8155b4e0c603 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:18.241506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 442/868] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:17 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 15:41:18.248760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:18.269864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:18.394915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58a831-2efe-477e-9049-76396647a54e None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:18.456891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3df835-8474-4000-b45f-92eaade6b7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:18.458305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df835-8474-4000-b45f-92eaade6b7a0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:18.458769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df835-8474-4000-b45f-92eaade6b7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:18.459278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3df835-8474-4000-b45f-92eaade6b7a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:18.479393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3df835-8474-4000-b45f-92eaade6b7a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:18.482095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3df835-8474-4000-b45f-92eaade6b7a0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-7f3df835-8474-4000-b45f-92eaade6b7a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:18.515096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:18.516036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:18.516913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:18.516913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:18.516913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:18.516913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:18.518464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 427/869] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:18.524700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e47ed-9c1f-4cd3-bcb2-8155b4e0c603 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:18.640409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:18.657324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e47ed-9c1f-4cd3-bcb2-8155b4e0c603 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:18.684459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68d38-08ca-4b57-8a56-26c1d09187df None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:18.684935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68d38-08ca-4b57-8a56-26c1d09187df None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:18.685808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad68d38-08ca-4b57-8a56-26c1d09187df None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:18.688705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 428/870] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:18.696228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e84c5e-1696-47bd-a546-b68940cf260f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:18.696718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e84c5e-1696-47bd-a546-b68940cf260f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:18.697006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e84c5e-1696-47bd-a546-b68940cf260f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:18.722254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-61e84c5e-1696-47bd-a546-b68940cf260f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:18.722586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:18.723152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:18.723438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:18.723714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:18.724347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-61e84c5e-1696-47bd-a546-b68940cf260f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:18.724543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:18.724885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:18.726400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:18.740278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 443/871] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:18.748348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:18.820533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:18.837860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 429/872] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:18.859398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e84c5e-1696-47bd-a546-b68940cf260f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:18.915985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c20869-3e86-419f-b167-bc2ca3358538 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:18.963301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:18.964661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:18.965026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:18.965370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:18.973329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:18.975155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:18.978666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:19.000529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:19.001733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:19.003510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:19.003510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:19.003510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:19.003510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:19.003510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:19.003510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:19.003510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:19.003510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:19.013667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 444/873] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:19.013939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e84c5e-1696-47bd-a546-b68940cf260f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.014877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.061092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79138c-4c2c-4d90-9e97-3d7614cbcb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:19.062748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79138c-4c2c-4d90-9e97-3d7614cbcb41 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.062748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79138c-4c2c-4d90-9e97-3d7614cbcb41 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.062748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79138c-4c2c-4d90-9e97-3d7614cbcb41 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.063161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79138c-4c2c-4d90-9e97-3d7614cbcb41 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:19.064468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79138c-4c2c-4d90-9e97-3d7614cbcb41 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-bc79138c-4c2c-4d90-9e97-3d7614cbcb41 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:19.078302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:19.079391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:19.080215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 430/874] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:19.080984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd9af0-4352-4be7-aa2c-6166db6a05ad None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.080984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd9af0-4352-4be7-aa2c-6166db6a05ad None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.081677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd9af0-4352-4be7-aa2c-6166db6a05ad None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.082815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 431/875] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:19.088927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8214222-2d65-49d2-9de8-2640135ba6bf None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.089512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8214222-2d65-49d2-9de8-2640135ba6bf None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.089512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8214222-2d65-49d2-9de8-2640135ba6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.107814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f8214222-2d65-49d2-9de8-2640135ba6bf None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:19.109009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f8214222-2d65-49d2-9de8-2640135ba6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:19.155466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a61af-6314-438b-a94d-bb847d66e3ae None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.170691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:19.199437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:19.200363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.200834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.201299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.201998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 432/876] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:19.202836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:19.203419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:19.204334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8214222-2d65-49d2-9de8-2640135ba6bf None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.206262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:19.234578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 445/877] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:19.235588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.322735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:19.367445 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8214222-2d65-49d2-9de8-2640135ba6bf None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.384325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bd3cb-8cfb-4740-b5a0-a6146cc64827 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.415593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:19.416310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.416584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.416855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.418118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:19.418428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:19.419846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:19.429503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 433/878] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:19.429819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:19.429995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.430113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.430113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.430113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.430113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:19.430113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:19.445191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:19.446377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:19.447282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 446/879] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:19.448132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.512364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:19.575908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51590379-8716-4b5e-bd4b-baed5082660d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.588632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe53b34-2414-4926-b806-dbed6d80312f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.612708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:19.613397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.613705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.613981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.614804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=fd307c87c08b42db9484dbb1c0d223e1) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:19.615219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:19.616632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:19.618172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:19.633803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 434/880] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/role_assignments?user.id=fd307c87c08b42db9484dbb1c0d223e1&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:19.633803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.637852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:19.638769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.639110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.639457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.640263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:19.640639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:19.642232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:19.655576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 447/881] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:19.656860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.789641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a42daf-27cc-41b0-8543-e20dea76dd28 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.790536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c191848-06fe-4081-8e2a-6023c5ecbab8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.813532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:19.833465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:19.833699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:19.834227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.834439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:19.834585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.834706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:19.834824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.835192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:19.835684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] RBAC: Adding query filter params (interface=public, service_id=c4ec2401631341cbafbc0366bef152f2, region_id=RegionOne) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:19.835845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:19.836075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:19.836179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:19.837429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:19.837544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:19.843266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 435/882] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:19.846636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 448/883] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/endpoints?service_id=c4ec2401631341cbafbc0366bef152f2&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:19.848896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.853264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:19.919591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:19.990224 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05a6af-0877-4b55-9558-41163d6099a9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:19.996221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcad68c-c203-4b7b-b864-c81f095efc78 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:20.033580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdba85b-4e16-4059-84cb-6eb40a3f9f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:20.034208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdba85b-4e16-4059-84cb-6eb40a3f9f5a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:20.034423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdba85b-4e16-4059-84cb-6eb40a3f9f5a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:20.034705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdba85b-4e16-4059-84cb-6eb40a3f9f5a None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:20.035694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdba85b-4e16-4059-84cb-6eb40a3f9f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:20.037876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdba85b-4e16-4059-84cb-6eb40a3f9f5a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:20.049858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 449/884] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/services => generated 1011 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:41:20.055685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624dc4d-81fe-4fb8-93f0-d85ad817f66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:20.056387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624dc4d-81fe-4fb8-93f0-d85ad817f66f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:20.056667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624dc4d-81fe-4fb8-93f0-d85ad817f66f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:20.056941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624dc4d-81fe-4fb8-93f0-d85ad817f66f None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:20.057662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624dc4d-81fe-4fb8-93f0-d85ad817f66f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:20.058000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624dc4d-81fe-4fb8-93f0-d85ad817f66f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:20.059419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624dc4d-81fe-4fb8-93f0-d85ad817f66f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:20.065436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 436/885] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:20.332624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbbf5c-cbf6-46a6-954a-82506973e702 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:20.332921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbbf5c-cbf6-46a6-954a-82506973e702 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:20.333212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbbf5c-cbf6-46a6-954a-82506973e702 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:20.335276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 450/886] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:20.341230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfd727-791e-4c52-a6a1-515381fe206c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:20.342101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfd727-791e-4c52-a6a1-515381fe206c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:20.342726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfd727-791e-4c52-a6a1-515381fe206c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:20.367804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-77cfd727-791e-4c52-a6a1-515381fe206c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:20.370016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-77cfd727-791e-4c52-a6a1-515381fe206c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:20.438442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:20.473650 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 437/887] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:20.502672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbbf5c-cbf6-46a6-954a-82506973e702 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:20.650382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbbf5c-cbf6-46a6-954a-82506973e702 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:20.668540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:20.687896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b0fb9-4e2b-4541-a56a-676de0a98750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:20.689141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0fb9-4e2b-4541-a56a-676de0a98750 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:20.689141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0fb9-4e2b-4541-a56a-676de0a98750 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:20.689141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0fb9-4e2b-4541-a56a-676de0a98750 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:20.689882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b0fb9-4e2b-4541-a56a-676de0a98750 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:20.692595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b0fb9-4e2b-4541-a56a-676de0a98750 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:20.710513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 451/888] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:41:20 2022] POST /identity/v3/services => generated 269 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:21.990970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfb0c8-c469-4bb1-bcbd-6a396826cbd6 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:21.991916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfb0c8-c469-4bb1-bcbd-6a396826cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:21.991916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfb0c8-c469-4bb1-bcbd-6a396826cbd6 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:21.994122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 438/889] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:22.001237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc42b2f-e8d7-44b2-a3a6-0fe631803db7 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.002050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc42b2f-e8d7-44b2-a3a6-0fe631803db7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.002209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc42b2f-e8d7-44b2-a3a6-0fe631803db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.028009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-cdc42b2f-e8d7-44b2-a3a6-0fe631803db7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:22.030055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cdc42b2f-e8d7-44b2-a3a6-0fe631803db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:22.115639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 452/890] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:22.138781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfb0c8-c469-4bb1-bcbd-6a396826cbd6 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:22.253415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfb0c8-c469-4bb1-bcbd-6a396826cbd6 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:22.290892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45556b8e-f6d0-4e53-b6f2-c3bc6afa1d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:22.292097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45556b8e-f6d0-4e53-b6f2-c3bc6afa1d1c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.292597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45556b8e-f6d0-4e53-b6f2-c3bc6afa1d1c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.293171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45556b8e-f6d0-4e53-b6f2-c3bc6afa1d1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.305980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45556b8e-f6d0-4e53-b6f2-c3bc6afa1d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-45556b8e-f6d0-4e53-b6f2-c3bc6afa1d1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:22.315034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:22.315981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:22.316799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:22.320400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 439/891] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jun 21 15:41:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:22.326794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc42b2f-e8d7-44b2-a3a6-0fe631803db7 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:22.452178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef525-bbfa-457f-aa7b-ebea2c57b9d3 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.452505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef525-bbfa-457f-aa7b-ebea2c57b9d3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.452743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ef525-bbfa-457f-aa7b-ebea2c57b9d3 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.454997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 440/892] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:22.460029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a87f0-a4a4-45bb-9c21-ba14f83f82a5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.460289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a87f0-a4a4-45bb-9c21-ba14f83f82a5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.460289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a87f0-a4a4-45bb-9c21-ba14f83f82a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.463922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc42b2f-e8d7-44b2-a3a6-0fe631803db7 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:22.465917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:22.481399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-422a87f0-a4a4-45bb-9c21-ba14f83f82a5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:22.482853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-422a87f0-a4a4-45bb-9c21-ba14f83f82a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:22.511276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:22.512276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.512746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.513802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.515051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:22.515550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:22.517866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:22.521249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:22.533067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 453/893] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jun 21 15:41:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:22.539499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:22.569499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 441/894] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:22.571322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a272b7-7005-487a-9536-31d4d5d78993 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.571625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a272b7-7005-487a-9536-31d4d5d78993 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.571907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a272b7-7005-487a-9536-31d4d5d78993 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.573321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 442/895] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:22.578453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258fdc4-b6cc-4921-aeaa-437cee584966 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.578733 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258fdc4-b6cc-4921-aeaa-437cee584966 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.579015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258fdc4-b6cc-4921-aeaa-437cee584966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.610237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d258fdc4-b6cc-4921-aeaa-437cee584966 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:22.612655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d258fdc4-b6cc-4921-aeaa-437cee584966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:22.659494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f12b9-0b5f-4fd1-bac8-3132096758be None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:22.696354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:22.697855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.697855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.702219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.702305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 443/896] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:22.702305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258fdc4-b6cc-4921-aeaa-437cee584966 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:22.714297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:22.716117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:22.741510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:22.742754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:22.743707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:22.743707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:22.743707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:22.743707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:22.743707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 454/897] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:22.743993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:22.769488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:22.848113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:22.860705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258fdc4-b6cc-4921-aeaa-437cee584966 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:22.895324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19be96-dda5-4312-9618-7a89c10a3ed9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:22.900999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:22.901948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.902297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.902636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.903363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:22.904805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:22.920116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:22.921170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 21 15:41:22.922139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:22.926255 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 444/898] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:22.926974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:22.938047 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186c73-3750-4f0a-a119-ff457c11e1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:22.938822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186c73-3750-4f0a-a119-ff457c11e1fe None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:22.939167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186c73-3750-4f0a-a119-ff457c11e1fe None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:22.939513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca186c73-3750-4f0a-a119-ff457c11e1fe None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:22.940318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca186c73-3750-4f0a-a119-ff457c11e1fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:22.941688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca186c73-3750-4f0a-a119-ff457c11e1fe None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:22.958387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 455/899] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:41:22 2022] POST /identity/v3/services => generated 241 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:22.959284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186c73-3750-4f0a-a119-ff457c11e1fe None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.061790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:23.066527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a4567-0b1e-4c22-b01c-53d4b1478c1b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:23.101430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca186c73-3750-4f0a-a119-ff457c11e1fe None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:23.106763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2843c7-37c1-4ac6-b18a-4918d5f94d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:23.107457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2843c7-37c1-4ac6-b18a-4918d5f94d17 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.107741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2843c7-37c1-4ac6-b18a-4918d5f94d17 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.108019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2843c7-37c1-4ac6-b18a-4918d5f94d17 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.108771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2843c7-37c1-4ac6-b18a-4918d5f94d17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:23.109074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2843c7-37c1-4ac6-b18a-4918d5f94d17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:23.110542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2843c7-37c1-4ac6-b18a-4918d5f94d17 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:23.131601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 445/900] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:23.131601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd36e8a-aa38-4feb-9a4a-44c9ea26109e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.131601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd36e8a-aa38-4feb-9a4a-44c9ea26109e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.131601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd36e8a-aa38-4feb-9a4a-44c9ea26109e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.131601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 446/901] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:23.134034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd36e8a-aa38-4feb-9a4a-44c9ea26109e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.152631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce32c0-206d-44c1-b623-61d3fe09a6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:23.153325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce32c0-206d-44c1-b623-61d3fe09a6be None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.153611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce32c0-206d-44c1-b623-61d3fe09a6be None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.153934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce32c0-206d-44c1-b623-61d3fe09a6be None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.154663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce32c0-206d-44c1-b623-61d3fe09a6be None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:23.154969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce32c0-206d-44c1-b623-61d3fe09a6be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:23.156368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce32c0-206d-44c1-b623-61d3fe09a6be None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:23.159137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:23.163556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 456/902] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:23.164941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2fcee-d636-4fd2-8768-a04fa1125df1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.165188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2fcee-d636-4fd2-8768-a04fa1125df1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.165445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2fcee-d636-4fd2-8768-a04fa1125df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.185186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c0a2fcee-d636-4fd2-8768-a04fa1125df1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:23.186453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c0a2fcee-d636-4fd2-8768-a04fa1125df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:23.286877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd36e8a-aa38-4feb-9a4a-44c9ea26109e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:23.294720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 457/903] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:23.294720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2fcee-d636-4fd2-8768-a04fa1125df1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.341754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:23.342546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.342809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.343083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.353128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:23.354630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:23.374923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:23.375979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:23.376956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:23.379975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 447/904] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:41:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:23.382025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.409315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:23.463231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2fcee-d636-4fd2-8768-a04fa1125df1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:23.507524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:23.508211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.508491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.508763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.509298 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:23.509482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:23.510120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:23.511629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:23.523354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 458/905] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Jun 21 15:41:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:23.524144 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.538999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27448c62-b910-4afd-a4ef-b03de4d51a7e None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:23.588067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:23.589741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.590505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.591280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.592822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:23.595433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:23.618656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:23.620571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:23.629875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 448/906] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Jun 21 15:41:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:23.632376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.666994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6637a9-0026-4acc-8416-d342258a9b31 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:23.700559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:23.706520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:23.707210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.707492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.707768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.708621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:23.708974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:23.710702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:23.730041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 459/907] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jun 21 15:41:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:23.730882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.752206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0660d-b240-4a54-98fb-b1e8a67bc4e1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:23.794160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6fce0-b5c7-4b4d-9016-6a6c98852d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:23.795134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6fce0-b5c7-4b4d-9016-6a6c98852d02 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.795565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6fce0-b5c7-4b4d-9016-6a6c98852d02 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.795974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6fce0-b5c7-4b4d-9016-6a6c98852d02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.796943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6fce0-b5c7-4b4d-9016-6a6c98852d02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:23.798495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6fce0-b5c7-4b4d-9016-6a6c98852d02 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:23.815746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:23.834703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 449/908] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:41:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:23.836658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6fce0-b5c7-4b4d-9016-6a6c98852d02 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.868433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5ec5d-76f5-496a-a668-54df9cbabd8f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:23.934230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:23.935090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:23.935380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:23.935660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:23.936382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:23.936694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:23.938783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:23.948856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 460/909] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:23.951572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:23.981097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6fce0-b5c7-4b4d-9016-6a6c98852d02 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:24.023072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59f060-98da-443c-8313-9c47e9f7e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:24.023997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59f060-98da-443c-8313-9c47e9f7e951 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:24.024630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59f060-98da-443c-8313-9c47e9f7e951 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:24.025000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59f060-98da-443c-8313-9c47e9f7e951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:24.025400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:24.096791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa76a09-5c31-4798-829f-0e59a810aaf1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:24.114477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:24.120237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59f060-98da-443c-8313-9c47e9f7e951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:24.121666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59f060-98da-443c-8313-9c47e9f7e951 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:24.155484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:24.157503 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:24.158419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:24.158849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:24.160925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:24.161537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:24.163515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:24.171007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 461/910] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:24.180752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:24.269473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 450/911] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:23 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:24.277810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59f060-98da-443c-8313-9c47e9f7e951 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:24.372345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463cf6c-dace-4892-a9dd-1365585a298f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:24.390342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:24.394928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:24.403555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59f060-98da-443c-8313-9c47e9f7e951 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:24.420067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:24.420752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:24.421038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:24.421319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:24.423787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] RBAC: Adding query filter params (interface=public, service_id=30b60ffab68e406eb6122c56c6a58c63, region_id=RegionOne) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:24.424114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:24.425595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:24.434718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 462/912] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jun 21 15:41:24 2022] GET /identity/v3/endpoints?service_id=30b60ffab68e406eb6122c56c6a58c63&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:24.440934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:24.467711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:24.468538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:24.468860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:24.470224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:24.491383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:24.494582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:24.528977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:24.530543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:24.532034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:24.532034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:24.532034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:24.532034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:24.533678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 451/913] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:24.542900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:24.580902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d86eb-070d-4b37-beed-3cc83673d3e8 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:24.626735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaf7ed-530d-43fd-8489-85048ee05c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:24.628550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaf7ed-530d-43fd-8489-85048ee05c71 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:24.629043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaf7ed-530d-43fd-8489-85048ee05c71 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:24.629531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaf7ed-530d-43fd-8489-85048ee05c71 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:24.630467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcaf7ed-530d-43fd-8489-85048ee05c71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:24.632098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcaf7ed-530d-43fd-8489-85048ee05c71 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:24.641456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 463/914] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:41:24 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:41:24.678535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015292fb-30f8-4176-a41a-9a66ede62ba0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:24.726394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:24.727094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:24.727372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:24.727644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:24.728448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:24.728826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:24.730318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:24.746725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 452/915] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:24.752658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaf7ed-530d-43fd-8489-85048ee05c71 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:24.811902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:24.888444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaf7ed-530d-43fd-8489-85048ee05c71 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:24.928484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:24.929295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:24.929618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:24.929932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:24.937466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:24.939053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:24.955446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:24.956827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:24.958204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 464/916] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:24.963078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:24.979010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:25.110096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9feb4-2147-4b6f-88d1-b2d7d7fb382b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:25.151567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390dd069-f4d6-4ac1-8f9d-0611f3938210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:25.153073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dd069-f4d6-4ac1-8f9d-0611f3938210 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:25.153558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dd069-f4d6-4ac1-8f9d-0611f3938210 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:25.154087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390dd069-f4d6-4ac1-8f9d-0611f3938210 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:25.155105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dd069-f4d6-4ac1-8f9d-0611f3938210 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:25.155614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dd069-f4d6-4ac1-8f9d-0611f3938210 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:25.157261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390dd069-f4d6-4ac1-8f9d-0611f3938210 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:25.171413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 453/917] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:25.177256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:25.298664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:25.305045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb210884-ba2e-4664-b392-e9b30f8e5ed6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:25.341897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b53e37-5fc9-4c67-a872-a763c9ea7805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:25.342566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b53e37-5fc9-4c67-a872-a763c9ea7805 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:25.342847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b53e37-5fc9-4c67-a872-a763c9ea7805 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:25.343117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b53e37-5fc9-4c67-a872-a763c9ea7805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:25.343955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b53e37-5fc9-4c67-a872-a763c9ea7805 None admin] RBAC: Adding query filter params (role.id=c255d4a547bc4efd818a3ecfcd75843f, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=fd307c87c08b42db9484dbb1c0d223e1) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:25.344260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b53e37-5fc9-4c67-a872-a763c9ea7805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:25.345700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b53e37-5fc9-4c67-a872-a763c9ea7805 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:25.353378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 465/918] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:25 2022] GET /identity/v3/role_assignments?user.id=fd307c87c08b42db9484dbb1c0d223e1&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=c255d4a547bc4efd818a3ecfcd75843f => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:25.407172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717846-2f68-4d57-91a1-a8465ac4372f None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:25.407826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717846-2f68-4d57-91a1-a8465ac4372f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:25.408321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717846-2f68-4d57-91a1-a8465ac4372f None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:25.410674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 454/919] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:25.417765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17895f5-5e29-4cf6-869c-6cd459aa4afa None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:25.418507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17895f5-5e29-4cf6-869c-6cd459aa4afa None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:25.418994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17895f5-5e29-4cf6-869c-6cd459aa4afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:25.438706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c17895f5-5e29-4cf6-869c-6cd459aa4afa None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:25.440307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c17895f5-5e29-4cf6-869c-6cd459aa4afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:25.522965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 466/920] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:25.547831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be717846-2f68-4d57-91a1-a8465ac4372f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:25.624936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:25.658998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be717846-2f68-4d57-91a1-a8465ac4372f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:25.698206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:25.699484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:25.700058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:25.700574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:25.701586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:25.703298 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:25.717375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:25.718422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:25.719239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:25.722258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 455/921] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:25.731692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17895f5-5e29-4cf6-869c-6cd459aa4afa None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:25.850034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:25.859612 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17895f5-5e29-4cf6-869c-6cd459aa4afa None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:25.911642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197f167-fdb3-4dae-98ce-6688c6c2949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:25.913135 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f167-fdb3-4dae-98ce-6688c6c2949b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:25.913397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f167-fdb3-4dae-98ce-6688c6c2949b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:25.913665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197f167-fdb3-4dae-98ce-6688c6c2949b None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:25.914478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197f167-fdb3-4dae-98ce-6688c6c2949b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:25.914881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197f167-fdb3-4dae-98ce-6688c6c2949b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:25.916888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197f167-fdb3-4dae-98ce-6688c6c2949b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:25.922936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 467/922] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:25.929690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:26.050308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4072d-5814-4f6b-ba44-367a0a88d19c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:26.091288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:26.092005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:26.092285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:26.092561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:26.093269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:26.093573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:26.095018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:26.096300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:26.101806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 456/923] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:26.101981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6aabf-2959-45bd-837d-0fffdd7b763c None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:26.102103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6aabf-2959-45bd-837d-0fffdd7b763c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:26.102484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6aabf-2959-45bd-837d-0fffdd7b763c None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:26.103941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 468/924] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:26.106779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:26.108829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd349-7154-4f82-8c11-09c09f743f87 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:26.109112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd349-7154-4f82-8c11-09c09f743f87 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:26.109388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fd349-7154-4f82-8c11-09c09f743f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:26.126799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c77fd349-7154-4f82-8c11-09c09f743f87 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:26.128158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c77fd349-7154-4f82-8c11-09c09f743f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:26.208412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 469/925] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:26.227141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d03362-993e-4fd3-a43d-97d2b58b6a39 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:26.235455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fd349-7154-4f82-8c11-09c09f743f87 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:26.282987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:26.283807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:26.284201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:26.284610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:26.285426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] RBAC: Adding query filter params (interface=public, service_id=f617a5526a6e40bc9d7d53c0ef0228d6, region_id=RegionOne) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:26.286024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:26.288247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:26.295539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 457/926] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jun 21 15:41:26 2022] GET /identity/v3/endpoints?service_id=f617a5526a6e40bc9d7d53c0ef0228d6&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:26.300699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:26.351975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:26.367031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:26.370928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fd349-7154-4f82-8c11-09c09f743f87 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:26.409762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ad048-615d-4fa8-8b8a-4b5e466b5d2d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:26.410861 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:26.411561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:26.412152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:26.412539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:26.413739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:26.414948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:26.416399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING keystone.server.flask.common [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 21 15:41:26.436630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.sql.core [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 15:41:26.436630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 15:41:26.436630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [parameters: {'id': 'e54787af350342c3b3ced01d47014aad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 15:41:26.436630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86658) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 15:41:26.448066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be321a3-8859-4144-9e82-d7bccf8e9504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:26.449052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be321a3-8859-4144-9e82-d7bccf8e9504 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:26.449490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be321a3-8859-4144-9e82-d7bccf8e9504 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:26.449928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be321a3-8859-4144-9e82-d7bccf8e9504 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:26.450749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be321a3-8859-4144-9e82-d7bccf8e9504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:26.452264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be321a3-8859-4144-9e82-d7bccf8e9504 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e54787af350342c3b3ced01d47014aad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e54787af350342c3b3ced01d47014aad. Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application conn.query(q) Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 15:41:26.456274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result.read() Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 21 15:41:26.457229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application next(self.gen) Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application yield resource Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application next(self.gen) Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise self.value Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application session.commit() Jun 21 15:41:26.458407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.session.flush() Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application compat.raise_( Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 15:41:26.459540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 15:41:26.461241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application conn.query(q) Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result.read() Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 15:41:26.462255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [parameters: {'id': 'e54787af350342c3b3ced01d47014aad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:26.463343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:26.464227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e54787af350342c3b3ced01d47014aad. Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 470/927] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Jun 21 15:41:26 2022] POST /identity/v3/projects => generated 262 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:26.465116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:26.469001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 458/928] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:41:26 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:41:26.625448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520734b7-eaeb-4be5-a62f-7ee780331a73 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:26.673113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216acf36-3b2f-4e86-8b88-63e581c96f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:26.674330 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216acf36-3b2f-4e86-8b88-63e581c96f5b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:26.674848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216acf36-3b2f-4e86-8b88-63e581c96f5b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:26.675269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216acf36-3b2f-4e86-8b88-63e581c96f5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:26.683650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216acf36-3b2f-4e86-8b88-63e581c96f5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:26.685720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216acf36-3b2f-4e86-8b88-63e581c96f5b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-216acf36-3b2f-4e86-8b88-63e581c96f5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:26.702661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:26.703792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:26.704662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:26.704662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:26.704662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:26.704662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:26.704662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:26.704662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:26.704662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:26.704662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:26.705376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 471/929] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:26.712473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be321a3-8859-4144-9e82-d7bccf8e9504 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:26.839323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be321a3-8859-4144-9e82-d7bccf8e9504 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:26.871359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:26.882321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c99e14-978f-4b88-8890-a36fc9defdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:26.883310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c99e14-978f-4b88-8890-a36fc9defdf6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:26.883844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c99e14-978f-4b88-8890-a36fc9defdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:26.884619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c99e14-978f-4b88-8890-a36fc9defdf6 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:26.885564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c99e14-978f-4b88-8890-a36fc9defdf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:26.886070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c99e14-978f-4b88-8890-a36fc9defdf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:26.887695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c99e14-978f-4b88-8890-a36fc9defdf6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:26.908258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 459/930] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:27.023321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f4953-4173-4280-a4fe-e0993312aff8 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.023552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f4953-4173-4280-a4fe-e0993312aff8 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.023773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f4953-4173-4280-a4fe-e0993312aff8 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.030411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 472/931] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:27.034372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f28547-f1c1-4f02-b7f7-9c79eba7e049 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.034668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f28547-f1c1-4f02-b7f7-9c79eba7e049 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.034957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f28547-f1c1-4f02-b7f7-9c79eba7e049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.057984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-42f28547-f1c1-4f02-b7f7-9c79eba7e049 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:27.059525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-42f28547-f1c1-4f02-b7f7-9c79eba7e049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:27.156658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 460/932] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:27.181839 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f4953-4173-4280-a4fe-e0993312aff8 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:27.186972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:27.311433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f4953-4173-4280-a4fe-e0993312aff8 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:27.350185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:27.351103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.351380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.351649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.352314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:27.353708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:27.370041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:27.371558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:27.373542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 473/933] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Jun 21 15:41:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:27.378792 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f28547-f1c1-4f02-b7f7-9c79eba7e049 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:27.487820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f28547-f1c1-4f02-b7f7-9c79eba7e049 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:27.510002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:27.529942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ad51-20da-45d4-a0fd-e7f55148e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:27.530681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ad51-20da-45d4-a0fd-e7f55148e6cc None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.530971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ad51-20da-45d4-a0fd-e7f55148e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.531272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ad51-20da-45d4-a0fd-e7f55148e6cc None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.531990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0ad51-20da-45d4-a0fd-e7f55148e6cc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:27.532302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0ad51-20da-45d4-a0fd-e7f55148e6cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:27.533687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0ad51-20da-45d4-a0fd-e7f55148e6cc None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:27.539289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 461/934] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:27.545201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:27.618281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:27.619162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67a487-92e0-4ec2-9085-b0a2915bf2b9 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.619452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67a487-92e0-4ec2-9085-b0a2915bf2b9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.619734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67a487-92e0-4ec2-9085-b0a2915bf2b9 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.623014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 462/935] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:41:27.627616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5a4a5-e0a9-4c62-90ac-308566602cce None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.627908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5a4a5-e0a9-4c62-90ac-308566602cce None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.628177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5a4a5-e0a9-4c62-90ac-308566602cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.645286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-86e5a4a5-e0a9-4c62-90ac-308566602cce None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:27.646776 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-86e5a4a5-e0a9-4c62-90ac-308566602cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:27.666890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05147589-21a0-4ae1-9b54-77eb4b0e6eb8 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:27.720329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:27.721019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.721304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.721582 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.722347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:27.722661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:27.724283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:27.733853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 474/936] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:27.737340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 463/937] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:27.738513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:27.769656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5a4a5-e0a9-4c62-90ac-308566602cce None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:27.867197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e9988-68cc-4c84-b94c-a1f5fbe1ed3f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:27.900117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5a4a5-e0a9-4c62-90ac-308566602cce None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:27.903074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:27.911451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17881d75-17d6-4cec-939d-623db7434a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:27.912528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17881d75-17d6-4cec-939d-623db7434a4e None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.913061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17881d75-17d6-4cec-939d-623db7434a4e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.913510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17881d75-17d6-4cec-939d-623db7434a4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.915002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17881d75-17d6-4cec-939d-623db7434a4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:27.916754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:27.918806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17881d75-17d6-4cec-939d-623db7434a4e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:27.948071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aab47f-3643-4463-bea3-7202a3775c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:27.949298 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab47f-3643-4463-bea3-7202a3775c46 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:27.949845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab47f-3643-4463-bea3-7202a3775c46 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:27.950327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab47f-3643-4463-bea3-7202a3775c46 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:27.961805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 475/938] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:41:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:27.973804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aab47f-3643-4463-bea3-7202a3775c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-88aab47f-3643-4463-bea3-7202a3775c46 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:27.989381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:27.990751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:27.991778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:27.992485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 464/939] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jun 21 15:41:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:28.003478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17881d75-17d6-4cec-939d-623db7434a4e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:28.150110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17881d75-17d6-4cec-939d-623db7434a4e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:28.205753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:28.207113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:28.207560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:28.208000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:28.209637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:28.209847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:28.211829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:28.226913 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 476/940] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jun 21 15:41:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:28.232981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aab47f-3643-4463-bea3-7202a3775c46 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:28.373891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aab47f-3643-4463-bea3-7202a3775c46 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:28.417485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:28.418383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:28.418668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:28.418943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:28.433251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:28.434797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:28.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:28.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:28.463465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:28.463465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:28.463465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:28.463465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:28.463692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 465/941] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:28.470167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:28.523502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:28.585190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd5d67-6f44-4868-88a1-30f10eb168c3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:28.622905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:28.623941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:28.624462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:28.624929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:28.625875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:28.626388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:28.627961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:28.642444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 477/942] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:28.648736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:28.675379 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:28.767992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312dbe5-22ee-4437-bdaa-3173cf97fc13 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:28.808589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e475c1-efa4-458b-bf31-be07aa055ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:28.809927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e475c1-efa4-458b-bf31-be07aa055ed9 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:28.810422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e475c1-efa4-458b-bf31-be07aa055ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:28.810898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e475c1-efa4-458b-bf31-be07aa055ed9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:28.820047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e475c1-efa4-458b-bf31-be07aa055ed9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:28.821879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e475c1-efa4-458b-bf31-be07aa055ed9 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-b0e475c1-efa4-458b-bf31-be07aa055ed9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:28.838165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:28.839144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:28.840141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:28.842865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 466/943] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:41:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:28.850248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:28.881093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17864-d785-4531-b948-567ab6b722a3 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:28.881508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17864-d785-4531-b948-567ab6b722a3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:28.881942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17864-d785-4531-b948-567ab6b722a3 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:28.883770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 467/944] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:28.890661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242eb6-177f-4300-b125-c2ae1f72b0b6 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:28.890950 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242eb6-177f-4300-b125-c2ae1f72b0b6 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:28.891233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242eb6-177f-4300-b125-c2ae1f72b0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:28.912153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-4e242eb6-177f-4300-b125-c2ae1f72b0b6 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:28.913865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-4e242eb6-177f-4300-b125-c2ae1f72b0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:28.963895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec394088-1d07-48d8-b142-98cd52857c54 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:28.972501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:28.989823 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 468/945] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:29.011442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:29.012443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.012928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.013464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.014556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:29.015034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:29.017357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242eb6-177f-4300-b125-c2ae1f72b0b6 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:29.020136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:29.034462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:29.046176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 478/946] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jun 21 15:41:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:29.051701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:29.153829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242eb6-177f-4300-b125-c2ae1f72b0b6 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:29.197420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:29.199046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.199681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.200389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.201706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:29.202783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3785fef-54e0-447f-b954-fa6f558f41ba None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:29.204743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:29.224480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:29.225796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:29.226634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:29.236006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 469/947] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:29.237720 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:29.277852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:29.279146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.279965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.280536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.300397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:29.304522 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:29.329819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:29.330954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:29.332048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:29.332048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:29.332048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:29.332048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:29.335204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 479/948] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:29.345824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:29.354748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:29.369316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:29.387000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6a1d5-8919-4dce-bccc-bb9b6179c5bf None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:29.425023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:29.426614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.427061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.427666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.428574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:29.428887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:29.430366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:29.440275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 470/949] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:29.442037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:29.485352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9256791-f33e-46c5-a11d-431fe508aa23 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:29.530871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:29.531484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.531765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.532168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.532943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:29.533442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:29.535409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:29.561723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 480/950] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:29.563546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:29.596079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cb82a-9329-4823-8da5-f85389857454 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:29.641707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89266bd1-59bb-4207-99e2-4a1259832475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:29.642425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89266bd1-59bb-4207-99e2-4a1259832475 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.642704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89266bd1-59bb-4207-99e2-4a1259832475 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.642983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89266bd1-59bb-4207-99e2-4a1259832475 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.644004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89266bd1-59bb-4207-99e2-4a1259832475 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:29.644319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89266bd1-59bb-4207-99e2-4a1259832475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:29.645710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89266bd1-59bb-4207-99e2-4a1259832475 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:29.651926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 471/951] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:29.653371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2a3c-669e-4920-896f-87eea0bdbdc5 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.653660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2a3c-669e-4920-896f-87eea0bdbdc5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.653960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf2a3c-669e-4920-896f-87eea0bdbdc5 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.657808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 472/952] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:29.659580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf2a3c-669e-4920-896f-87eea0bdbdc5 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:29.664964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:29.683023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:29.724433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7478e95-e7cb-4a8f-ad65-89af9bfc9146 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:29.765138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773c83a-0b28-4a63-8f81-eb6f8b6dabfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:29.766509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773c83a-0b28-4a63-8f81-eb6f8b6dabfe None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.767018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773c83a-0b28-4a63-8f81-eb6f8b6dabfe None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.767552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773c83a-0b28-4a63-8f81-eb6f8b6dabfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.776337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773c83a-0b28-4a63-8f81-eb6f8b6dabfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:29.777820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773c83a-0b28-4a63-8f81-eb6f8b6dabfe None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:29.782902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf2a3c-669e-4920-896f-87eea0bdbdc5 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-e773c83a-0b28-4a63-8f81-eb6f8b6dabfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:29.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:29.797339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 481/953] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742fc2-d6fa-40dd-a941-168f4932f2c2 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.802225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742fc2-d6fa-40dd-a941-168f4932f2c2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.804336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742fc2-d6fa-40dd-a941-168f4932f2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.815453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b9742fc2-d6fa-40dd-a941-168f4932f2c2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:29.816913 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b9742fc2-d6fa-40dd-a941-168f4932f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:29.839646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c58e33-5479-4a10-aee6-d1c68196617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:29.840338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c58e33-5479-4a10-aee6-d1c68196617f None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:29.840620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c58e33-5479-4a10-aee6-d1c68196617f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:29.840895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c58e33-5479-4a10-aee6-d1c68196617f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:29.841896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c58e33-5479-4a10-aee6-d1c68196617f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:29.843724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c58e33-5479-4a10-aee6-d1c68196617f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:29.885548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 473/954] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:41:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:41:29.885548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c58e33-5479-4a10-aee6-d1c68196617f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:29.959050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 482/955] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:29.982380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:30.001625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742fc2-d6fa-40dd-a941-168f4932f2c2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:30.050549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c58e33-5479-4a10-aee6-d1c68196617f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:30.095104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:30.096163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:30.096604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:30.097040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:30.098618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:30.099344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:30.103026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:30.125728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 474/956] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:30.131705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:30.133762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:30.179854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742fc2-d6fa-40dd-a941-168f4932f2c2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:30.226530 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:30.228306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:30.229098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:30.229595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:30.230813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:30.233065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:30.260650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.sql.core [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 15:41:30.260650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 15:41:30.260650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [parameters: {'user_id': '8e546356be724813b50acecc7e7da2be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 15:41:30.260650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86658) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application conn.query(q) Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result.read() Jun 21 15:41:30.269306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 15:41:30.270359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application next(self.gen) Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application yield resource Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application next(self.gen) Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.session.rollback() Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.force_reraise() Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise self.value Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application session.commit() Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 21 15:41:30.271999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._prepare_impl() Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.session.flush() Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._flush(objects) Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application compat.raise_( Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application flush_context.execute() Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rec.execute(self) Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 21 15:41:30.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 21 15:41:30.275014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result = self._query(query) Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application conn.query(q) Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application result.read() Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 21 15:41:30.277551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [parameters: {'user_id': '8e546356be724813b50acecc7e7da2be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:30.280159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 21 15:41:30.282199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 21 15:41:30.284583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 21 15:41:30.284583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 21 15:41:30.284583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 21 15:41:30.284583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 21 15:41:30.284583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 21 15:41:30.284583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:30.284583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 483/957] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Jun 21 15:41:29 2022] POST /identity/v3/users => generated 160 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:30.284583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:30.301931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8be047-66ae-446d-938a-f229a4f07d76 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:30.334518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:30.362363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe951e-48fe-4d88-b84c-8cfd0f9f79f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:30.363732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe951e-48fe-4d88-b84c-8cfd0f9f79f7 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:30.364217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe951e-48fe-4d88-b84c-8cfd0f9f79f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:30.364982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe951e-48fe-4d88-b84c-8cfd0f9f79f7 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/fd307c87c08b42db9484dbb1c0d223e1/roles/c255d4a547bc4efd818a3ecfcd75843f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:30.367850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe951e-48fe-4d88-b84c-8cfd0f9f79f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=fd307c87c08b42db9484dbb1c0d223e1, role_id=c255d4a547bc4efd818a3ecfcd75843f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:30.369763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe951e-48fe-4d88-b84c-8cfd0f9f79f7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:30.393738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 475/958] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:41:30 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/fd307c87c08b42db9484dbb1c0d223e1/roles/c255d4a547bc4efd818a3ecfcd75843f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:41:30.431845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b590c-5401-4317-b4bb-ab7695e661bc None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:30.460887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:30.474123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:30.475234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:30.475568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:30.475886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:30.487751 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:30.489531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:30.498743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385b0c6-0ed1-46e5-8032-bcc8d6287935 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:30.500055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385b0c6-0ed1-46e5-8032-bcc8d6287935 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:30.500705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385b0c6-0ed1-46e5-8032-bcc8d6287935 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:30.510058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 476/959] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:30.511750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:30.513276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:30.515697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 484/960] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:30.519890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d960ed0-3a01-4521-b4c7-54af2427238c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:30.520381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d960ed0-3a01-4521-b4c7-54af2427238c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:30.520845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d960ed0-3a01-4521-b4c7-54af2427238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:30.521882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:30.557508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1d960ed0-3a01-4521-b4c7-54af2427238c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:30.559873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1d960ed0-3a01-4521-b4c7-54af2427238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:30.674653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 477/961] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:30.696979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c8ea8-a211-483a-b2f8-edc83c6d7ef5 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:30.698335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d960ed0-3a01-4521-b4c7-54af2427238c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:30.744908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:30.746096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:30.746616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:30.747132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:30.748602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:30.749314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:30.751774 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:30.809973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 485/962] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Jun 21 15:41:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:30.833754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d960ed0-3a01-4521-b4c7-54af2427238c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:30.848804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:30.880514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:30.881750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:30.882075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:30.882352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:30.883051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:30.885609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:30.904768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:30.905912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:30.906820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:30.907674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 478/963] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:30.913866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:30.978507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:31.041593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09383929-d004-46c3-b7ee-633927a9c8ef None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:31.081053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd489b90-fa47-4fca-a27c-6c8eedde4089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:31.081728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd489b90-fa47-4fca-a27c-6c8eedde4089 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:31.082036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd489b90-fa47-4fca-a27c-6c8eedde4089 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:31.082311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd489b90-fa47-4fca-a27c-6c8eedde4089 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:31.083029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd489b90-fa47-4fca-a27c-6c8eedde4089 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:31.083363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd489b90-fa47-4fca-a27c-6c8eedde4089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:31.085181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd489b90-fa47-4fca-a27c-6c8eedde4089 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:31.095588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 486/964] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:31.097411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:31.209024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b8ade-65d8-4893-a909-64bba09fa144 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:31.258960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b04117-fae7-4f43-96b2-87aa6d10d3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:31.259770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b04117-fae7-4f43-96b2-87aa6d10d3a3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:31.260059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b04117-fae7-4f43-96b2-87aa6d10d3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:31.260337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b04117-fae7-4f43-96b2-87aa6d10d3a3 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:31.261108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b04117-fae7-4f43-96b2-87aa6d10d3a3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:31.261415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b04117-fae7-4f43-96b2-87aa6d10d3a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:31.263027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b04117-fae7-4f43-96b2-87aa6d10d3a3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:31.269877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 479/965] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:32.199125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e388262-ab7d-481e-9cd4-44587f49c91b None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:32.199717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e388262-ab7d-481e-9cd4-44587f49c91b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:32.199717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e388262-ab7d-481e-9cd4-44587f49c91b None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:32.201041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 487/966] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:32.206988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1372-c72a-4b6a-8472-31870bf70091 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:32.207455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1372-c72a-4b6a-8472-31870bf70091 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:32.207556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc1372-c72a-4b6a-8472-31870bf70091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:32.224708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-96bc1372-c72a-4b6a-8472-31870bf70091 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:32.225945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-96bc1372-c72a-4b6a-8472-31870bf70091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:32.267046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:32.321870 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 480/967] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:32.343700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e388262-ab7d-481e-9cd4-44587f49c91b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:32.453425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e388262-ab7d-481e-9cd4-44587f49c91b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:32.461143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:32.491124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa78aff-24d9-4732-8e76-3942af21e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:32.492082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa78aff-24d9-4732-8e76-3942af21e4bb None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:32.492362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa78aff-24d9-4732-8e76-3942af21e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:32.492646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa78aff-24d9-4732-8e76-3942af21e4bb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:32.493499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa78aff-24d9-4732-8e76-3942af21e4bb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:32.494877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa78aff-24d9-4732-8e76-3942af21e4bb None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-afa78aff-24d9-4732-8e76-3942af21e4bb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:32.508755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:32.510073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:32.511797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:32.514150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 488/968] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:32.516215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc1372-c72a-4b6a-8472-31870bf70091 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:32.634859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc1372-c72a-4b6a-8472-31870bf70091 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:32.672530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:32.673208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:32.673490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:32.673766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:32.674505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:32.674813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:32.676200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:32.681868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 481/969] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:32.687133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:32.687606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef55783-093e-4bb2-b021-7ca798fb5fda None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:32.687743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef55783-093e-4bb2-b021-7ca798fb5fda None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:32.687914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef55783-093e-4bb2-b021-7ca798fb5fda None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:32.689460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 489/970] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:32.694792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2031f5-e88f-4e4a-928f-cabd32fa3247 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:32.695078 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2031f5-e88f-4e4a-928f-cabd32fa3247 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:32.695350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2031f5-e88f-4e4a-928f-cabd32fa3247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:32.714209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-4f2031f5-e88f-4e4a-928f-cabd32fa3247 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:32.715636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4f2031f5-e88f-4e4a-928f-cabd32fa3247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:32.726428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:32.792114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 490/971] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:32.807244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af92e7-0b2d-48d3-bfba-38fe2ab6633c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:32.816192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2031f5-e88f-4e4a-928f-cabd32fa3247 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:32.847358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:32.848034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:32.848316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:32.848588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:32.849568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:32.849911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:32.851289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:32.856782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 482/972] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:32.862694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:32.934108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:32.937805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2031f5-e88f-4e4a-928f-cabd32fa3247 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:32.985569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25872ca4-735f-4f2b-a028-1a567766a560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:32.986529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872ca4-735f-4f2b-a028-1a567766a560 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:32.986950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872ca4-735f-4f2b-a028-1a567766a560 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:32.987098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872ca4-735f-4f2b-a028-1a567766a560 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:32.993142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:33.000225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ad11b-c71d-42fe-9637-bf37affea719 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:33.002928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25872ca4-735f-4f2b-a028-1a567766a560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-25872ca4-735f-4f2b-a028-1a567766a560 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:33.011156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:33.012087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:33.012929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 491/973] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jun 21 15:41:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:33.018746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25872ca4-735f-4f2b-a028-1a567766a560 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:33.048689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:33.049407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.049695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.050114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.050979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] RBAC: Adding query filter params (interface=admin, service_id=f617a5526a6e40bc9d7d53c0ef0228d6, region_id=RegionOne) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:33.051309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:33.053060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:33.059164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 483/974] 10.222.0.52 () {56 vars in 1249 bytes} [Tue Jun 21 15:41:32 2022] GET /identity/v3/endpoints?service_id=f617a5526a6e40bc9d7d53c0ef0228d6&interface=admin®ion_id=RegionOne => generated 193 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:33.064857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:33.152652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25872ca4-735f-4f2b-a028-1a567766a560 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:33.196630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:33.197320 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.197608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.197923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.198677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:33.198982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:33.200375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:33.201246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de830a81-e6c1-4e5d-8c30-fc8d2ffaf83d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:33.210575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 492/975] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jun 21 15:41:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:33.216682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:33.252451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:33.252891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4d778-224b-484b-8106-444102c97a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:33.253572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4d778-224b-484b-8106-444102c97a1f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.253869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4d778-224b-484b-8106-444102c97a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.254142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4d778-224b-484b-8106-444102c97a1f None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.254825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4d778-224b-484b-8106-444102c97a1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:33.256388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4d778-224b-484b-8106-444102c97a1f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:33.269877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 484/976] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:41:33 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:41:33.358703 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99a798-0656-4cc6-aa46-c03a1e072b13 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:33.409761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:33.410658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.410942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.411211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.416942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5b805-284b-44e4-8297-32e11d6d474c None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.417309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5b805-284b-44e4-8297-32e11d6d474c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.417558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5b805-284b-44e4-8297-32e11d6d474c None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.419325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 485/977] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:33.425022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e129dbf-c345-45c7-abb2-f917312fd545 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.425309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e129dbf-c345-45c7-abb2-f917312fd545 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.425659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e129dbf-c345-45c7-abb2-f917312fd545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.426418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:33.428141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:33.447550 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2e129dbf-c345-45c7-abb2-f917312fd545 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:33.447784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:33.448629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:33.449570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:33.449570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:33.449570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:33.449570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:33.449817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2e129dbf-c345-45c7-abb2-f917312fd545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:33.449907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 493/978] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:33.455434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:33.471570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:33.539691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 486/979] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:33.551972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:33.564773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e129dbf-c345-45c7-abb2-f917312fd545 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:33.588609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0800b-aab6-4788-9595-5693dc7ac491 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:33.646492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:33.647419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.647694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.647995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.648706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:33.649125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:33.650611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:33.664738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 494/980] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:33.671579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:33.689766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e129dbf-c345-45c7-abb2-f917312fd545 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:33.732489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bce187-489d-4305-af53-ab9cc9350c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:33.733139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bce187-489d-4305-af53-ab9cc9350c52 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.733421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bce187-489d-4305-af53-ab9cc9350c52 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.733697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bce187-489d-4305-af53-ab9cc9350c52 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.734462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bce187-489d-4305-af53-ab9cc9350c52 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:33.735848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bce187-489d-4305-af53-ab9cc9350c52 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:33.754880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 487/981] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:41:33 2022] POST /identity/v3/services => generated 242 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:33.810729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f4b7-00ee-47d6-a4e1-38415313cd76 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:33.854891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:33.867616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:33.869740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:33.870412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:33.870904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:33.881109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:33.882627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:33.899757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:33.901195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:33.903161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 495/982] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:41:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:33.907291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bce187-489d-4305-af53-ab9cc9350c52 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:33.929197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:34.027338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bce187-489d-4305-af53-ab9cc9350c52 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:34.064671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:34.065261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:34.065539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:34.065833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:34.066689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:34.066991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:34.068458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:34.086616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 488/983] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Jun 21 15:41:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:34.092190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:34.194695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722a5df-2c72-4851-96d5-e416203e72d3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:34.230955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39432ca0-5e2a-4ab3-bd59-b12945af0c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:34.231602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39432ca0-5e2a-4ab3-bd59-b12945af0c87 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:34.231873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39432ca0-5e2a-4ab3-bd59-b12945af0c87 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:34.232146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39432ca0-5e2a-4ab3-bd59-b12945af0c87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:34.301799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:34.315614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39432ca0-5e2a-4ab3-bd59-b12945af0c87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:34.317006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39432ca0-5e2a-4ab3-bd59-b12945af0c87 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:34.428175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 496/984] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:34 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:34.433888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:34.507616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:34.540512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f759e7-aae9-481f-bca3-a0e2bb512726 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:34.578876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:34.579597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:34.579881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:34.580159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:34.593210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:34.594694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:34.614940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:34.621005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:34.622145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:34.622145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:34.622145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:34.622145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:34.622145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 489/985] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:34.622433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39432ca0-5e2a-4ab3-bd59-b12945af0c87 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:34.721504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:34.732931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39432ca0-5e2a-4ab3-bd59-b12945af0c87 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:34.772344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:34.773021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:34.773280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:34.773574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:34.774462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:34.774768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:34.776160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:34.792810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 497/986] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:34.793993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:34.908947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cd4d1-bec0-4034-a100-c1cc6c715fd4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:34.937291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:34.950790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:34.951602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:34.951876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:34.952152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:34.960653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:34.962128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:34.976994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:34.978601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:34.980327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:34.980327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:34.980327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:34.980327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:34.980327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:34.980327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:34.980327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:34.980327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:34.983403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 490/987] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:34.984181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:35.097351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e447cff-e0c8-4337-ac09-73895899be16 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:35.133415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba69ac-f74f-491c-950b-935edf08a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:35.134119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba69ac-f74f-491c-950b-935edf08a339 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:35.134403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba69ac-f74f-491c-950b-935edf08a339 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:35.134679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba69ac-f74f-491c-950b-935edf08a339 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:35.135479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba69ac-f74f-491c-950b-935edf08a339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:35.135788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba69ac-f74f-491c-950b-935edf08a339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:35.137252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba69ac-f74f-491c-950b-935edf08a339 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:35.150815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 498/988] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:35.156795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:35.275293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eab4e-c4c0-4eb2-afde-34c389c9fdc8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:35.311828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffa40d-6b06-4be4-8a5c-d89202a12ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:35.312809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffa40d-6b06-4be4-8a5c-d89202a12ac1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:35.313266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffa40d-6b06-4be4-8a5c-d89202a12ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:35.313742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffa40d-6b06-4be4-8a5c-d89202a12ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:35.314781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffa40d-6b06-4be4-8a5c-d89202a12ac1 None admin] RBAC: Adding query filter params (role.id=c255d4a547bc4efd818a3ecfcd75843f, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=fd307c87c08b42db9484dbb1c0d223e1) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:35.315321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffa40d-6b06-4be4-8a5c-d89202a12ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:35.317034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffa40d-6b06-4be4-8a5c-d89202a12ac1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:35.324776 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 491/989] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:35 2022] GET /identity/v3/role_assignments?user.id=fd307c87c08b42db9484dbb1c0d223e1&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=c255d4a547bc4efd818a3ecfcd75843f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:35.342814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776483e8-3cd4-4710-bdf1-973af7d66b09 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:35.343343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776483e8-3cd4-4710-bdf1-973af7d66b09 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:35.343458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776483e8-3cd4-4710-bdf1-973af7d66b09 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:35.344831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 499/990] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:35.350894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370fc23-59a7-42ce-aca0-ad3b4908829c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:35.351222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370fc23-59a7-42ce-aca0-ad3b4908829c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:35.351524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370fc23-59a7-42ce-aca0-ad3b4908829c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:35.369675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-9370fc23-59a7-42ce-aca0-ad3b4908829c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:35.370935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9370fc23-59a7-42ce-aca0-ad3b4908829c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:35.424988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:35.453172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 492/991] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:35.476274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776483e8-3cd4-4710-bdf1-973af7d66b09 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:35.489083 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:35.594512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776483e8-3cd4-4710-bdf1-973af7d66b09 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:35.631409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7cbec-f9d1-4dc7-b9cc-fe91d5128115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:35.632226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7cbec-f9d1-4dc7-b9cc-fe91d5128115 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:35.632505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7cbec-f9d1-4dc7-b9cc-fe91d5128115 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:35.632778 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7cbec-f9d1-4dc7-b9cc-fe91d5128115 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:35.633441 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7cbec-f9d1-4dc7-b9cc-fe91d5128115 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:35.635578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7cbec-f9d1-4dc7-b9cc-fe91d5128115 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-d3c7cbec-f9d1-4dc7-b9cc-fe91d5128115 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:35.649317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:35.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:35.651289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 500/992] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Jun 21 15:41:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:35.657275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370fc23-59a7-42ce-aca0-ad3b4908829c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:35.781800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370fc23-59a7-42ce-aca0-ad3b4908829c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:35.819734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:35.820399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:35.820679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:35.820951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:35.821661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:35.822149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:35.823618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:35.829622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 493/993] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:35.830655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ed4f9-7f25-4338-9631-824d3ca1caef None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:35.831119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ed4f9-7f25-4338-9631-824d3ca1caef None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:35.831402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ed4f9-7f25-4338-9631-824d3ca1caef None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:35.832925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 501/994] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:41:35.835330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:35.838264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78c7f-60d6-4889-8bca-39c022c714ad None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:35.838552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78c7f-60d6-4889-8bca-39c022c714ad None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:35.838829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78c7f-60d6-4889-8bca-39c022c714ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:35.858335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-68c78c7f-60d6-4889-8bca-39c022c714ad None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:35.859498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-68c78c7f-60d6-4889-8bca-39c022c714ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:35.899615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:35.943499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 502/995] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:35.970018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c78c7f-60d6-4889-8bca-39c022c714ad None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:35.973858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:35.981870 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a696c3d-1485-46f0-aef8-d63d9d5fe6cf None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:36.031371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:36.032489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:36.032973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:36.033449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:36.034795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:36.035335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:36.037951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:36.046495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 494/996] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Jun 21 15:41:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:36.052817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:36.096245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c78c7f-60d6-4889-8bca-39c022c714ad None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:36.141989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:36.143163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:36.143439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:36.143763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:36.144351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:36.146590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:36.165686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:36.167410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:36.168630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 503/997] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:36.173584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:36.190344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66cf82-f74e-4372-bd20-d204d0ed9cd3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:36.200286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:36.244367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2daeba-902f-48f7-9f7d-d52eb89467ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:36.245330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2daeba-902f-48f7-9f7d-d52eb89467ae None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:36.248241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2daeba-902f-48f7-9f7d-d52eb89467ae None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:36.248340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2daeba-902f-48f7-9f7d-d52eb89467ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:36.249282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2daeba-902f-48f7-9f7d-d52eb89467ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:36.251111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2daeba-902f-48f7-9f7d-d52eb89467ae None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:36.264232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:36.271435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 495/998] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:41:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:36.315863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6661e8-f1de-485d-9bd6-e2d9f532d569 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:36.370182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:36.371128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:36.371471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:36.371754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:36.372544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:36.372843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:36.374421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:36.381829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 504/999] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:36.387671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2daeba-902f-48f7-9f7d-d52eb89467ae None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:36.520930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2daeba-902f-48f7-9f7d-d52eb89467ae None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:36.561785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:36.562506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:36.562788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:36.563065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:36.563796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:36.564105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:36.565539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:36.571699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 496/1000] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:36.577988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:36.695457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c3fdd-1c07-43ea-bb80-715b54558ed4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:36.723343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:36.742275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2634a-2dae-446f-8895-9690dff82df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:36.742975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2634a-2dae-446f-8895-9690dff82df9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:36.743259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2634a-2dae-446f-8895-9690dff82df9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:36.743536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2634a-2dae-446f-8895-9690dff82df9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:36.744346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2634a-2dae-446f-8895-9690dff82df9 None admin] RBAC: Adding query filter params (interface=public, service_id=a080b126bcf24922926cd929c566c0af, region_id=RegionOne) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:36.744663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2634a-2dae-446f-8895-9690dff82df9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:36.746099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2634a-2dae-446f-8895-9690dff82df9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:36.756907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 505/1001] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jun 21 15:41:36 2022] GET /identity/v3/endpoints?service_id=a080b126bcf24922926cd929c566c0af&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:36.758174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:36.791652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:36.873240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e45f1-adf1-4b1b-9d01-9a7bb4f4bf60 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:36.911680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202bf65-3963-4ecc-9259-af598f219242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:36.912684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202bf65-3963-4ecc-9259-af598f219242 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:36.913008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202bf65-3963-4ecc-9259-af598f219242 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:36.913369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202bf65-3963-4ecc-9259-af598f219242 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:36.914133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202bf65-3963-4ecc-9259-af598f219242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:36.915526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202bf65-3963-4ecc-9259-af598f219242 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:36.922934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 497/1002] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:41:36 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:41:37.546304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fca1f8-ce65-40e7-8708-69219aa84e57 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:37.546849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fca1f8-ce65-40e7-8708-69219aa84e57 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:37.546849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fca1f8-ce65-40e7-8708-69219aa84e57 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:37.548345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 506/1003] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:37.554362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ccec6-39d6-4953-8305-dcc7e5abde3a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:37.554819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ccec6-39d6-4953-8305-dcc7e5abde3a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:37.555006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ccec6-39d6-4953-8305-dcc7e5abde3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:37.575451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c33ccec6-39d6-4953-8305-dcc7e5abde3a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:37.576696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c33ccec6-39d6-4953-8305-dcc7e5abde3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:37.656818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 498/1004] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:37 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:37.681796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fca1f8-ce65-40e7-8708-69219aa84e57 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:37.809594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fca1f8-ce65-40e7-8708-69219aa84e57 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:37.853037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:37.853895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:37.854508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:37.854747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] PATH_INFO: `/v3/services/image` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:37.855362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:37.856811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:37.869204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:37.870555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:37.871417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:37.874155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 507/1005] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Jun 21 15:41:37 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:37.876680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ccec6-39d6-4953-8305-dcc7e5abde3a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:37.904518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:37.994219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ccec6-39d6-4953-8305-dcc7e5abde3a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:38.034443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b39578-ec1f-416c-8958-a36d5e7ecddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:38.035115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b39578-ec1f-416c-8958-a36d5e7ecddf None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:38.035397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b39578-ec1f-416c-8958-a36d5e7ecddf None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:38.035669 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b39578-ec1f-416c-8958-a36d5e7ecddf None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:38.036595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b39578-ec1f-416c-8958-a36d5e7ecddf None admin] RBAC: Adding query filter params (name=image) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:38.036902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b39578-ec1f-416c-8958-a36d5e7ecddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:38.038516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b39578-ec1f-416c-8958-a36d5e7ecddf None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:38.044244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 499/1006] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:38.049677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:38.123066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:38.183162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d18d34-e425-426d-aef9-71d959c10835 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:38.221094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c5190-b992-43be-83c2-f6bd8747eefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:38.222086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5190-b992-43be-83c2-f6bd8747eefd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:38.222516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5190-b992-43be-83c2-f6bd8747eefd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:38.222939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5190-b992-43be-83c2-f6bd8747eefd None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:38.223932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c5190-b992-43be-83c2-f6bd8747eefd None admin] RBAC: Adding query filter params (type=image) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:38.224414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c5190-b992-43be-83c2-f6bd8747eefd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:38.226248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c5190-b992-43be-83c2-f6bd8747eefd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:38.232778 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 508/1007] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:38.504208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912d03e-83d5-4bac-97da-c568450fc6d3 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:38.504644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912d03e-83d5-4bac-97da-c568450fc6d3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:38.505075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912d03e-83d5-4bac-97da-c568450fc6d3 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:38.507203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 500/1008] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:38.512366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121990-6522-4829-85a8-9857bb051e59 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:38.512736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121990-6522-4829-85a8-9857bb051e59 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:38.513091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121990-6522-4829-85a8-9857bb051e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:38.531304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ed121990-6522-4829-85a8-9857bb051e59 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:38.532507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ed121990-6522-4829-85a8-9857bb051e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:38.607487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 509/1009] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:38.633474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912d03e-83d5-4bac-97da-c568450fc6d3 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:38.742451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:38.757115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912d03e-83d5-4bac-97da-c568450fc6d3 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:38.794264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c007b-7c6d-4e0a-8169-43b0c9f7b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:38.795076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c007b-7c6d-4e0a-8169-43b0c9f7b26b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:38.795355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c007b-7c6d-4e0a-8169-43b0c9f7b26b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:38.795641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c007b-7c6d-4e0a-8169-43b0c9f7b26b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:38.796851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c007b-7c6d-4e0a-8169-43b0c9f7b26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:38.798312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c007b-7c6d-4e0a-8169-43b0c9f7b26b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:38.800172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 501/1010] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:38.806025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121990-6522-4829-85a8-9857bb051e59 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:38.859485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:38.916406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed121990-6522-4829-85a8-9857bb051e59 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:38.955241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279cd5e-63b9-4909-a301-241af68a2163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:38.955924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279cd5e-63b9-4909-a301-241af68a2163 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:38.956183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279cd5e-63b9-4909-a301-241af68a2163 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:38.956492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279cd5e-63b9-4909-a301-241af68a2163 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:38.957230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279cd5e-63b9-4909-a301-241af68a2163 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:38.958754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279cd5e-63b9-4909-a301-241af68a2163 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:38.985877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 510/1011] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:41:38 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:39.059865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98bae-f94e-435b-964b-0933b946c444 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:39.060056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98bae-f94e-435b-964b-0933b946c444 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:39.060312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98bae-f94e-435b-964b-0933b946c444 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:39.061893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 502/1012] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:39.068073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88ad05-82e8-4f11-b264-5defb14925d3 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:39.068391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88ad05-82e8-4f11-b264-5defb14925d3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:39.068652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88ad05-82e8-4f11-b264-5defb14925d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:39.088613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-fd88ad05-82e8-4f11-b264-5defb14925d3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:39.089858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-fd88ad05-82e8-4f11-b264-5defb14925d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:39.170628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 511/1013] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:39.194439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c98bae-f94e-435b-964b-0933b946c444 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:39.313962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c98bae-f94e-435b-964b-0933b946c444 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:39.350628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:39.351527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:39.351702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:39.351966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] PATH_INFO: `/v3/services/compute` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:39.352607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:39.353996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:39.366868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:39.368028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 21 15:41:39.368859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:39.369595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 503/1014] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:39.375867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88ad05-82e8-4f11-b264-5defb14925d3 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:39.410662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:39.485704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88ad05-82e8-4f11-b264-5defb14925d3 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:39.523826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7239441-101f-4539-8a39-aee769465eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:39.524620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7239441-101f-4539-8a39-aee769465eef None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:39.524895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7239441-101f-4539-8a39-aee769465eef None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:39.525164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7239441-101f-4539-8a39-aee769465eef None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:39.525894 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7239441-101f-4539-8a39-aee769465eef None admin] RBAC: Adding query filter params (name=compute) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:39.526200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7239441-101f-4539-8a39-aee769465eef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:39.527569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7239441-101f-4539-8a39-aee769465eef None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:39.532778 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 512/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:39.540477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:39.568105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:39.652678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abed1fb-3783-4a7b-b091-f18dda6d826e None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:39.687692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc55f-61c7-44bd-bb9d-0d371a631cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:39.688367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc55f-61c7-44bd-bb9d-0d371a631cb6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:39.688647 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc55f-61c7-44bd-bb9d-0d371a631cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:39.688920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc55f-61c7-44bd-bb9d-0d371a631cb6 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:39.689629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fc55f-61c7-44bd-bb9d-0d371a631cb6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:39.689964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fc55f-61c7-44bd-bb9d-0d371a631cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:39.691337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897fc55f-61c7-44bd-bb9d-0d371a631cb6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:39.701456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 504/1016] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Jun 21 15:41:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:39.702598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7239441-101f-4539-8a39-aee769465eef None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:39.802611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:39.810092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7239441-101f-4539-8a39-aee769465eef None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:39.845489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc6944-ebc9-4a0b-96ba-787f266e8d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:39.846096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6944-ebc9-4a0b-96ba-787f266e8d11 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:39.846379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6944-ebc9-4a0b-96ba-787f266e8d11 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:39.846654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6944-ebc9-4a0b-96ba-787f266e8d11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:39.847410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc6944-ebc9-4a0b-96ba-787f266e8d11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:39.849159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc6944-ebc9-4a0b-96ba-787f266e8d11 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:39.877816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 513/1017] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:41:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:41:40.387574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8579bf-1f97-4ad7-9aae-7a81f1ef0813 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:40.387904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8579bf-1f97-4ad7-9aae-7a81f1ef0813 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:40.388128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8579bf-1f97-4ad7-9aae-7a81f1ef0813 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:40.389757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 505/1018] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:40.395972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787858-521e-47d0-ba59-459dbcfa20bf None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:40.396452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787858-521e-47d0-ba59-459dbcfa20bf None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:40.396868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787858-521e-47d0-ba59-459dbcfa20bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:40.420508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-db787858-521e-47d0-ba59-459dbcfa20bf None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:40.421871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-db787858-521e-47d0-ba59-459dbcfa20bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:40.502980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 514/1019] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:40 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:40.526238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8579bf-1f97-4ad7-9aae-7a81f1ef0813 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:40.621696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:40.643231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8579bf-1f97-4ad7-9aae-7a81f1ef0813 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:40.685665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdf1ed-6796-430a-8031-d8bdf70eaa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:40.686474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdf1ed-6796-430a-8031-d8bdf70eaa07 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:40.686661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdf1ed-6796-430a-8031-d8bdf70eaa07 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:40.686917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdf1ed-6796-430a-8031-d8bdf70eaa07 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:40.687635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdf1ed-6796-430a-8031-d8bdf70eaa07 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:40.689018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdf1ed-6796-430a-8031-d8bdf70eaa07 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:40.704466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 506/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:41:40 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:41.098478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ce6aa-07aa-439f-8b37-02e66b222f13 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:41.099203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ce6aa-07aa-439f-8b37-02e66b222f13 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:41.099203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ce6aa-07aa-439f-8b37-02e66b222f13 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:41.100611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 515/1021] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:41.106054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfc60e-bf3a-45cb-9e6a-758738d5d700 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:41.106420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfc60e-bf3a-45cb-9e6a-758738d5d700 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:41.106773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfc60e-bf3a-45cb-9e6a-758738d5d700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:41.125156 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-afdfc60e-bf3a-45cb-9e6a-758738d5d700 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:41.126679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-afdfc60e-bf3a-45cb-9e6a-758738d5d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:41.209655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 507/1022] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:41.234487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ce6aa-07aa-439f-8b37-02e66b222f13 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:41.322226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:41.347369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ce6aa-07aa-439f-8b37-02e66b222f13 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:41.388989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec2078-5864-44eb-b4d8-25dd3e154187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:41.389658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec2078-5864-44eb-b4d8-25dd3e154187 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:41.390090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec2078-5864-44eb-b4d8-25dd3e154187 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:41.390337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec2078-5864-44eb-b4d8-25dd3e154187 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:41.392901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec2078-5864-44eb-b4d8-25dd3e154187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:41.394429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec2078-5864-44eb-b4d8-25dd3e154187 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:41.401249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 516/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:41.402632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfc60e-bf3a-45cb-9e6a-758738d5d700 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:41.480624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:41.515605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdfc60e-bf3a-45cb-9e6a-758738d5d700 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:41.553269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c9eb0-95d6-4b3f-a7cf-efc3cf177b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:41.553981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c9eb0-95d6-4b3f-a7cf-efc3cf177b59 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:41.554383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c9eb0-95d6-4b3f-a7cf-efc3cf177b59 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:41.554629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c9eb0-95d6-4b3f-a7cf-efc3cf177b59 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:41.555733 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c9eb0-95d6-4b3f-a7cf-efc3cf177b59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:41.558200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c9eb0-95d6-4b3f-a7cf-efc3cf177b59 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:41.585814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 508/1024] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:41:41 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:42.053553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7a9dc-3774-4ae6-bc3e-3741e0f70c70 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:42.054023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7a9dc-3774-4ae6-bc3e-3741e0f70c70 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:42.054529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7a9dc-3774-4ae6-bc3e-3741e0f70c70 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:42.056752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 517/1025] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:42.063560 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b9b5c-5b19-4b8a-91bc-2ea36fa69f7c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:42.064070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b9b5c-5b19-4b8a-91bc-2ea36fa69f7c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:42.064161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b9b5c-5b19-4b8a-91bc-2ea36fa69f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:42.081752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ee5b9b5c-5b19-4b8a-91bc-2ea36fa69f7c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:42.083466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ee5b9b5c-5b19-4b8a-91bc-2ea36fa69f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:42.159500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 509/1026] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:42.183146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7a9dc-3774-4ae6-bc3e-3741e0f70c70 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:42.308700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7a9dc-3774-4ae6-bc3e-3741e0f70c70 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:42.346956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bb406-e3d7-4da2-9b5b-31eb31f01ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:42.347774 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bb406-e3d7-4da2-9b5b-31eb31f01ffe None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:42.348050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bb406-e3d7-4da2-9b5b-31eb31f01ffe None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:42.348469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bb406-e3d7-4da2-9b5b-31eb31f01ffe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:42.361111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bb406-e3d7-4da2-9b5b-31eb31f01ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-d32bb406-e3d7-4da2-9b5b-31eb31f01ffe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:42.370447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:42.371756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:42.372585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 518/1027] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jun 21 15:41:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:42.379879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b9b5c-5b19-4b8a-91bc-2ea36fa69f7c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:42.443801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:42.492989 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b9b5c-5b19-4b8a-91bc-2ea36fa69f7c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:42.529523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:42.530254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:42.530522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:42.530793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:42.531546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:42.531852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:42.533246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:42.552278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 510/1028] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jun 21 15:41:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:42.557841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bb406-e3d7-4da2-9b5b-31eb31f01ffe None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:42.567843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:42.664417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bb406-e3d7-4da2-9b5b-31eb31f01ffe None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:42.699988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad87e1-9948-44d6-adc7-c2eb14858e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:42.700797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad87e1-9948-44d6-adc7-c2eb14858e13 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:42.701080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad87e1-9948-44d6-adc7-c2eb14858e13 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:42.701355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad87e1-9948-44d6-adc7-c2eb14858e13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:42.713857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad87e1-9948-44d6-adc7-c2eb14858e13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:42.715310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad87e1-9948-44d6-adc7-c2eb14858e13 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-82ad87e1-9948-44d6-adc7-c2eb14858e13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:42.732737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:42.734100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:42.735059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:42.735059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:42.735059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:42.735059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:42.737866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 519/1029] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:42.739982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:42.778566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ef955-9cfa-4af1-80fa-a61c9a09e517 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:42.778909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ef955-9cfa-4af1-80fa-a61c9a09e517 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:42.779122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ef955-9cfa-4af1-80fa-a61c9a09e517 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:42.780656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 520/1030] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:41:42.786047 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d689-c4c9-4704-a1c2-357126e21760 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:42.786325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d689-c4c9-4704-a1c2-357126e21760 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:42.786596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9d689-c4c9-4704-a1c2-357126e21760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:42.806240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a2b9d689-c4c9-4704-a1c2-357126e21760 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:42.807439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a2b9d689-c4c9-4704-a1c2-357126e21760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:42.853897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f20512-738a-4b15-9b89-2149cefc9457 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:42.871494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:42.880066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:42.893813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 521/1031] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:42.907764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:42.908441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:42.908725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:42.909000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:42.909977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:42.910288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:42.912110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:42.915703 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9d689-c4c9-4704-a1c2-357126e21760 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:42.930933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 511/1032] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:42.937355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.041986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9d689-c4c9-4704-a1c2-357126e21760 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.058188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fe4d3-2041-4f19-acd5-0873a2ad5ac8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.079168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948798f-689a-424d-8597-c87b6532201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.079970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948798f-689a-424d-8597-c87b6532201a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.080247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948798f-689a-424d-8597-c87b6532201a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.080522 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948798f-689a-424d-8597-c87b6532201a None admin] PATH_INFO: `/v3/services/image` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.081690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948798f-689a-424d-8597-c87b6532201a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.083468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948798f-689a-424d-8597-c87b6532201a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.096906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-3948798f-689a-424d-8597-c87b6532201a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:43.097579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:43.098783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:43.099831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:43.101283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.101283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.101283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.101475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 522/1033] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Jun 21 15:41:42 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:43.105426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948798f-689a-424d-8597-c87b6532201a None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.111749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.113215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:43.128672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:43.130173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:43.131695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 512/1034] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:41:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:43.136360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.186079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:43.191114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:43.248500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948798f-689a-424d-8597-c87b6532201a None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.273584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c7914-1419-4ae0-86df-cee1f56a38f2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.298188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.299033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.299651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.299939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.300835 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] RBAC: Adding query filter params (name=image) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:43.301178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.302892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.309112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 523/1035] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:43.313674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.314730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.315739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.316012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.316286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.317170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:43.317480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.318962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.342915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 513/1036] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:43.349520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.446788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f56bce-8006-4a18-a104-d8f987be9f95 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.459472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:43.465702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e936f1-da8a-40bd-87ac-319cfcabf505 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.470283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:43.490654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.491365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.491761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.492115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.492904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] RBAC: Adding query filter params (type=image) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:43.493412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.494859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.504679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3da3-5ea6-4ac2-be43-14c0714f00e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.505437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 524/1037] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:43.505918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3da3-5ea6-4ac2-be43-14c0714f00e9 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.506267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3da3-5ea6-4ac2-be43-14c0714f00e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.506428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.506731 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3da3-5ea6-4ac2-be43-14c0714f00e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.592903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3da3-5ea6-4ac2-be43-14c0714f00e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.594523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3da3-5ea6-4ac2-be43-14c0714f00e9 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.624606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45db09f-5882-4437-a4af-5bb48684b200 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.660933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.661716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.662090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.662382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.663184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] RBAC: Adding query filter params (interface=public, service_id=43adf8bc0c484999894c5e0ad588e9d6, region_id=RegionOne) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:43.663494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.664877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.674764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 525/1038] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/endpoints?service_id=43adf8bc0c484999894c5e0ad588e9d6&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:43.676314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.702684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:43.711118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 514/1039] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 15:41:43.719469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3da3-5ea6-4ac2-be43-14c0714f00e9 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.737348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:43.805140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cd165-3cb1-431e-9833-a7f9b3a3216e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.828781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3da3-5ea6-4ac2-be43-14c0714f00e9 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:43.843776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f6831-1de1-42e2-9fe2-7ea7a1c356b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.844458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f6831-1de1-42e2-9fe2-7ea7a1c356b2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.844741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f6831-1de1-42e2-9fe2-7ea7a1c356b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.845017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f6831-1de1-42e2-9fe2-7ea7a1c356b2 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.845707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f6831-1de1-42e2-9fe2-7ea7a1c356b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.847121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f6831-1de1-42e2-9fe2-7ea7a1c356b2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.854208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 526/1040] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/services => generated 1957 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:41:43.855479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190f2b-1a69-4218-87b8-4913fdb9ab7d None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.855770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190f2b-1a69-4218-87b8-4913fdb9ab7d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.856043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd190f2b-1a69-4218-87b8-4913fdb9ab7d None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.857297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 527/1041] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:43.865218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb59d-c45b-4be9-a918-129320fa8c90 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.865699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb59d-c45b-4be9-a918-129320fa8c90 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.866214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb59d-c45b-4be9-a918-129320fa8c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.871954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:43.872779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:43.873071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:43.873346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:43.885877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-72cdb59d-c45b-4be9-a918-129320fa8c90 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:43.887102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-72cdb59d-c45b-4be9-a918-129320fa8c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:43.891004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:43.892804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:43.913738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:43.914961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:43.916053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:43.916053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:43.916053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:43.916053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:43.917881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 515/1042] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:43.923786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:43.975525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 528/1043] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:43.993409 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:44.000326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdb59d-c45b-4be9-a918-129320fa8c90 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:44.061988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676008a-561b-44d5-8f24-a31f01a559a0 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:44.062685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:44.106620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:44.107302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:44.107582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:44.107883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:44.108624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:44.108931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:44.110360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:44.122802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 516/1044] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:44.129281 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:44.135414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdb59d-c45b-4be9-a918-129320fa8c90 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:44.177291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ddf48-54b6-4cc7-b04d-a62195997be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:44.179315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ddf48-54b6-4cc7-b04d-a62195997be5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:44.179744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ddf48-54b6-4cc7-b04d-a62195997be5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:44.180157 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ddf48-54b6-4cc7-b04d-a62195997be5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:44.194382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12ddf48-54b6-4cc7-b04d-a62195997be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-f12ddf48-54b6-4cc7-b04d-a62195997be5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:44.203019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:44.204154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:44.205084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 529/1045] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:44.210600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ddf48-54b6-4cc7-b04d-a62195997be5 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:44.268877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878a192-8001-4c52-ae5d-8c98a5e948d9 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:44.280337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:44.312508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:44.313754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:44.314256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:44.314721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:44.322739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:44.324218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:44.324479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:44.333533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ddf48-54b6-4cc7-b04d-a62195997be5 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:44.340661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:44.341579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:44.342529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 517/1046] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:44.348192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:44.378042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:44.378708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:44.378989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:44.379546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:44.380369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:44.380876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:44.382780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:44.392844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 530/1047] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:44.400154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:44.470902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aef4c-42b2-46e6-917d-802a8c7b643b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:44.510043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:44.511289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:44.516103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:44.516103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:44.516103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:44.516103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:44.517992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:44.537514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ff8c6-8259-4ffc-a8f0-8fa622b773f1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:44.538248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 518/1048] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:44.547350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:44.593726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:44.594454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cde8fb-e419-4e87-9460-36783f52b7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:44.595752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cde8fb-e419-4e87-9460-36783f52b7ea None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:44.596283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cde8fb-e419-4e87-9460-36783f52b7ea None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:44.597025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cde8fb-e419-4e87-9460-36783f52b7ea None admin] PATH_INFO: `/v3/users/86b90e32fb894a749f21ed4fe3b633a6` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:44.622506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cde8fb-e419-4e87-9460-36783f52b7ea None admin] RBAC: Authorizing `identity:get_user(user_id=86b90e32fb894a749f21ed4fe3b633a6)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:44.625072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cde8fb-e419-4e87-9460-36783f52b7ea None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:44.632406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 531/1049] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:41:44 2022] GET /identity/v3/users/86b90e32fb894a749f21ed4fe3b633a6 => generated 285 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:44.633629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cde8fb-e419-4e87-9460-36783f52b7ea None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:44.665321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:44.701315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d1b34-8ba1-4982-9aab-cb86021d2aa7 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:44.755122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:44.755881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:44.756086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:44.756350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:44.757200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] RBAC: Adding query filter params (role.id=c255d4a547bc4efd818a3ecfcd75843f, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=46e5ca80b7834ab7bc951b454c9aae49) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:44.757624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:44.759753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:44.766570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 519/1050] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:44 2022] GET /identity/v3/role_assignments?user.id=46e5ca80b7834ab7bc951b454c9aae49&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=c255d4a547bc4efd818a3ecfcd75843f => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:44.772353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cde8fb-e419-4e87-9460-36783f52b7ea None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:44.816375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef84f1d-7d85-469e-a098-9e01742ec0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:44.817377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef84f1d-7d85-469e-a098-9e01742ec0b8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:44.817377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef84f1d-7d85-469e-a098-9e01742ec0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:44.817632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef84f1d-7d85-469e-a098-9e01742ec0b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:44.916834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef84f1d-7d85-469e-a098-9e01742ec0b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:44.918778 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef84f1d-7d85-469e-a098-9e01742ec0b8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:44.940790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:45.048269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 532/1051] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:45.049745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:45.160488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:45.171424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46d35-17bc-49fc-a346-2afd83f061ea None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:45.209249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51602810-0fff-4ecf-bdf4-c6ef72c541bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:45.210175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51602810-0fff-4ecf-bdf4-c6ef72c541bc None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:45.210417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51602810-0fff-4ecf-bdf4-c6ef72c541bc None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:45.210708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51602810-0fff-4ecf-bdf4-c6ef72c541bc None admin] PATH_INFO: `/v3/projects/70b433a900fd4fc3a35b39031d787992` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:45.211956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51602810-0fff-4ecf-bdf4-c6ef72c541bc None admin] RBAC: Authorizing `identity:get_project(project_id=70b433a900fd4fc3a35b39031d787992)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:45.213541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51602810-0fff-4ecf-bdf4-c6ef72c541bc None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:45.215550 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 520/1052] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:41:45 2022] GET /identity/v3/projects/70b433a900fd4fc3a35b39031d787992 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:45.221579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef84f1d-7d85-469e-a098-9e01742ec0b8 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:45.367061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef84f1d-7d85-469e-a098-9e01742ec0b8 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:45.386777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:45.415895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465acf25-b824-45c5-a747-2f162f321ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:45.416993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465acf25-b824-45c5-a747-2f162f321ad7 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:45.417475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465acf25-b824-45c5-a747-2f162f321ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:45.417983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465acf25-b824-45c5-a747-2f162f321ad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:45.419535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465acf25-b824-45c5-a747-2f162f321ad7 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=70b433a900fd4fc3a35b39031d787992, user.id=86b90e32fb894a749f21ed4fe3b633a6) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:45.420352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465acf25-b824-45c5-a747-2f162f321ad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:45.423188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465acf25-b824-45c5-a747-2f162f321ad7 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:45.441816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 533/1053] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:45 2022] GET /identity/v3/role_assignments?user.id=86b90e32fb894a749f21ed4fe3b633a6&scope.project.id=70b433a900fd4fc3a35b39031d787992&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:45.968624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecc58e-523c-4b69-bb6a-2992933b21ca None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:45.968968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecc58e-523c-4b69-bb6a-2992933b21ca None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:45.969198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecc58e-523c-4b69-bb6a-2992933b21ca None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:45.970895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 521/1054] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:45.976903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883bb2c-ccfb-4a78-8bae-ecd6beec8fc5 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:45.977186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883bb2c-ccfb-4a78-8bae-ecd6beec8fc5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:45.977469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883bb2c-ccfb-4a78-8bae-ecd6beec8fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:45.996629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-7883bb2c-ccfb-4a78-8bae-ecd6beec8fc5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:45.998325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7883bb2c-ccfb-4a78-8bae-ecd6beec8fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:46.094256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 534/1055] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:46.112043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecc58e-523c-4b69-bb6a-2992933b21ca None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:46.252924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecc58e-523c-4b69-bb6a-2992933b21ca None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:46.291318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:46.292369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:46.292822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:46.293237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] PATH_INFO: `/v3/services/image` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:46.294063 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:46.295591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:46.312643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:46.313721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 21 15:41:46.314585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:46.315296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 522/1056] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Jun 21 15:41:46 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:46.321112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883bb2c-ccfb-4a78-8bae-ecd6beec8fc5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:46.432162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883bb2c-ccfb-4a78-8bae-ecd6beec8fc5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:46.447367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:46.471706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:46.472654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:46.473878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:46.474299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:46.475178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] RBAC: Adding query filter params (name=image) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:46.475623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:46.477135 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:46.483061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 535/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:46.489274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:46.518503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:46.610353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d9ed4-398b-4d28-a7b7-076a654bc86a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:46.653585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79259df9-74a1-4d3f-9538-f156b65f46a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:46.654364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259df9-74a1-4d3f-9538-f156b65f46a5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:46.654772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259df9-74a1-4d3f-9538-f156b65f46a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:46.655075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259df9-74a1-4d3f-9538-f156b65f46a5 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:46.655962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79259df9-74a1-4d3f-9538-f156b65f46a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:46.656295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79259df9-74a1-4d3f-9538-f156b65f46a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:46.658159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79259df9-74a1-4d3f-9538-f156b65f46a5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:46.667301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 523/1058] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:46.673675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:46.788692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349bfe9-5411-4d8b-81e2-451a7c05b695 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:46.825355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ed291-0949-48a7-b340-bcbd3a152106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:46.826104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ed291-0949-48a7-b340-bcbd3a152106 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:46.826387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ed291-0949-48a7-b340-bcbd3a152106 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:46.826704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ed291-0949-48a7-b340-bcbd3a152106 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:46.827459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ed291-0949-48a7-b340-bcbd3a152106 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:46.828833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ed291-0949-48a7-b340-bcbd3a152106 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:46.853991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 536/1059] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:41:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:46.858439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25696b-047b-435d-87d5-2b99e8619041 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:46.858743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25696b-047b-435d-87d5-2b99e8619041 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:46.859022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25696b-047b-435d-87d5-2b99e8619041 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:46.861744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 524/1060] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:46.867228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557fb46-a6f6-4694-8e3c-2977511b00eb None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:46.867514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557fb46-a6f6-4694-8e3c-2977511b00eb None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:46.867984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557fb46-a6f6-4694-8e3c-2977511b00eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:46.887225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-4557fb46-a6f6-4694-8e3c-2977511b00eb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:46.888468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4557fb46-a6f6-4694-8e3c-2977511b00eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:46.922033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:46.979604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 537/1061] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:47.002557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25696b-047b-435d-87d5-2b99e8619041 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:47.111791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:47.126845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25696b-047b-435d-87d5-2b99e8619041 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:47.173090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d0024-d95d-4844-abea-c478e347d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:47.173970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d0024-d95d-4844-abea-c478e347d300 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:47.174256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d0024-d95d-4844-abea-c478e347d300 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:47.174530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d0024-d95d-4844-abea-c478e347d300 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:47.187662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d0024-d95d-4844-abea-c478e347d300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-a89d0024-d95d-4844-abea-c478e347d300 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:47.196061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:47.197056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:47.197961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 525/1062] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jun 21 15:41:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:47.204810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557fb46-a6f6-4694-8e3c-2977511b00eb None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:47.322100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557fb46-a6f6-4694-8e3c-2977511b00eb None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:47.361633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29f6e9-00fd-4d9d-95ac-492a63271272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:47.362419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29f6e9-00fd-4d9d-95ac-492a63271272 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:47.362647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29f6e9-00fd-4d9d-95ac-492a63271272 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:47.362927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29f6e9-00fd-4d9d-95ac-492a63271272 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:47.363833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29f6e9-00fd-4d9d-95ac-492a63271272 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:47.364140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29f6e9-00fd-4d9d-95ac-492a63271272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:47.365537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29f6e9-00fd-4d9d-95ac-492a63271272 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:47.379763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 538/1063] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jun 21 15:41:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:47.380630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d0024-d95d-4844-abea-c478e347d300 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:47.499911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d0024-d95d-4844-abea-c478e347d300 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:47.525681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5c8b-1933-4edd-8b0e-133fee94e29b None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:47.526001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5c8b-1933-4edd-8b0e-133fee94e29b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:47.526282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c5c8b-1933-4edd-8b0e-133fee94e29b None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:47.527841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 539/1064] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:41:47.532915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e26031-01e7-4766-b63a-ac94c74406c3 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:47.533201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e26031-01e7-4766-b63a-ac94c74406c3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:47.533473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e26031-01e7-4766-b63a-ac94c74406c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:47.543315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:47.544120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:47.544406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:47.544682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:47.550498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d0e26031-01e7-4766-b63a-ac94c74406c3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:47.551868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d0e26031-01e7-4766-b63a-ac94c74406c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:47.556706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:47.558205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:47.567673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:47.567997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:47.579265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:47.580219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:47.581230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:47.581230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:47.581230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:47.581230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:47.581230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 526/1065] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:47.589898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:47.641164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 540/1066] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:47.664488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e26031-01e7-4766-b63a-ac94c74406c3 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:47.721901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b42193-e6e4-44a7-bc95-e0c0b21a1da4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:47.766487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:47.767830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:47.768296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:47.768778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:47.769871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:47.770443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:47.772049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:47.783568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e26031-01e7-4766-b63a-ac94c74406c3 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:47.785906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 527/1067] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:47.799756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:47.839461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd692e80-d4c0-4436-bfac-4bad8f525e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:47.840278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd692e80-d4c0-4436-bfac-4bad8f525e9f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:47.840558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd692e80-d4c0-4436-bfac-4bad8f525e9f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:47.840832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd692e80-d4c0-4436-bfac-4bad8f525e9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:47.853243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd692e80-d4c0-4436-bfac-4bad8f525e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-bd692e80-d4c0-4436-bfac-4bad8f525e9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:47.861386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:47.864477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:47.865367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 541/1068] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:47.868069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:47.869689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd692e80-d4c0-4436-bfac-4bad8f525e9f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:47.890672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:47.918208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03069be1-fc08-4f50-bee5-2f4cbf5280d5 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:47.954850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:47.956045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:47.956542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:47.957073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:47.965857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:47.967591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:47.983285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:47.984431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:47.985597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:47.987977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 528/1069] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:41:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:47.993551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:48.037155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd692e80-d4c0-4436-bfac-4bad8f525e9f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.090166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:48.090865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:48.091143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:48.091413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:48.092222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:48.092516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:48.093989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:48.104300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 542/1070] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:48.110057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:48.137098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3281b-8cd8-4fc3-80f8-1cb9ad3faa40 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.162056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:48.183564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:48.184421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:48.184714 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:48.185132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:48.186333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:48.186960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:48.188889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:48.198097 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:48.215326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 529/1071] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:41:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:48.225448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:48.241727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a072b-934b-45f1-84f5-2725fe397cec None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.282798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76609803-0afb-4045-b43d-3d0c5d66b0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:48.283691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76609803-0afb-4045-b43d-3d0c5d66b0cf None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:48.283977 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76609803-0afb-4045-b43d-3d0c5d66b0cf None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:48.284604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76609803-0afb-4045-b43d-3d0c5d66b0cf None admin] PATH_INFO: `/v3/users/86b90e32fb894a749f21ed4fe3b633a6` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:48.285968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76609803-0afb-4045-b43d-3d0c5d66b0cf None admin] RBAC: Authorizing `identity:get_user(user_id=86b90e32fb894a749f21ed4fe3b633a6)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:48.287899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76609803-0afb-4045-b43d-3d0c5d66b0cf None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:48.289925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 543/1072] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:41:48 2022] GET /identity/v3/users/86b90e32fb894a749f21ed4fe3b633a6 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:48.295334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76609803-0afb-4045-b43d-3d0c5d66b0cf None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:48.353996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bc0f7-c153-4a8d-85af-11819f78e336 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.393565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:48.394575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:48.394951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:48.395237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:48.402200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76609803-0afb-4045-b43d-3d0c5d66b0cf None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.407611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:48.409084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:48.428806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:48.429794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:48.430693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:48.430693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:48.430693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:48.430693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:48.430693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 530/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:48.436112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:48.449445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db598f16-a24f-41da-9f55-0c04be62a29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:48.450484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db598f16-a24f-41da-9f55-0c04be62a29b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:48.450777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db598f16-a24f-41da-9f55-0c04be62a29b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:48.451047 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db598f16-a24f-41da-9f55-0c04be62a29b None admin] PATH_INFO: `/v3/projects/70b433a900fd4fc3a35b39031d787992` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:48.452331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db598f16-a24f-41da-9f55-0c04be62a29b None admin] RBAC: Authorizing `identity:get_project(project_id=70b433a900fd4fc3a35b39031d787992)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:48.453874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db598f16-a24f-41da-9f55-0c04be62a29b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:48.455754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 544/1074] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:41:48 2022] GET /identity/v3/projects/70b433a900fd4fc3a35b39031d787992 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:48.461657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db598f16-a24f-41da-9f55-0c04be62a29b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:48.468433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:48.476373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:48.559929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c96ff-8172-4ceb-a850-c309d2a08e2d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.586201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db598f16-a24f-41da-9f55-0c04be62a29b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.600023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:48.600783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:48.601064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:48.601339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:48.602092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:48.602396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:48.603932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:48.621604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 531/1075] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:48.623595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:48.635102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1f489-bfb5-4957-a08a-607b61d6313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:48.636088 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1f489-bfb5-4957-a08a-607b61d6313f None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:48.636372 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1f489-bfb5-4957-a08a-607b61d6313f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:48.636662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1f489-bfb5-4957-a08a-607b61d6313f None admin] PATH_INFO: `/v3/projects/70b433a900fd4fc3a35b39031d787992/users/86b90e32fb894a749f21ed4fe3b633a6/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:48.639031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1f489-bfb5-4957-a08a-607b61d6313f None admin] RBAC: Authorizing `identity:create_grant(project_id=70b433a900fd4fc3a35b39031d787992, user_id=86b90e32fb894a749f21ed4fe3b633a6, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:48.640620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1f489-bfb5-4957-a08a-607b61d6313f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:48.661883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 545/1076] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:41:48 2022] PUT /identity/v3/projects/70b433a900fd4fc3a35b39031d787992/users/86b90e32fb894a749f21ed4fe3b633a6/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:41:48.756636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:48.768487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4cf67-1875-4e78-a976-836bfbdbcb67 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.806922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:48.807770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:48.808055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:48.808336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:48.817030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:48.818532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:48.834677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:48.835621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:48.836537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 532/1077] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:48.843458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1f489-bfb5-4957-a08a-607b61d6313f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:48.922307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:48.960751 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1f489-bfb5-4957-a08a-607b61d6313f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:48.999417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2da301-10fd-480c-9129-2a7a3e7c9b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:49.000103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da301-10fd-480c-9129-2a7a3e7c9b3b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:49.000381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da301-10fd-480c-9129-2a7a3e7c9b3b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:49.000655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2da301-10fd-480c-9129-2a7a3e7c9b3b None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:49.001491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2da301-10fd-480c-9129-2a7a3e7c9b3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:49.001862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2da301-10fd-480c-9129-2a7a3e7c9b3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:49.003395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2da301-10fd-480c-9129-2a7a3e7c9b3b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:49.018115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 546/1078] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:49.024843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:49.042033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af234fd-9e5f-4949-9dc8-d1ce5d06600c None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:49.042380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af234fd-9e5f-4949-9dc8-d1ce5d06600c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:49.042586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af234fd-9e5f-4949-9dc8-d1ce5d06600c None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:49.044252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 547/1079] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:41:49.049877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5be17-9747-460a-b025-b06ccd393a38 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:49.050175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5be17-9747-460a-b025-b06ccd393a38 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:49.050580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5be17-9747-460a-b025-b06ccd393a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:49.069667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a6e5be17-9747-460a-b025-b06ccd393a38 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:49.071681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a6e5be17-9747-460a-b025-b06ccd393a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:49.157819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 548/1080] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:49.168773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21b2e2-c30c-4fd9-9a67-554e50cf067c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:49.177096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5be17-9747-460a-b025-b06ccd393a38 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:49.218326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a2e9f-9ee4-41f1-88b6-7016d4cd4bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:49.219378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a2e9f-9ee4-41f1-88b6-7016d4cd4bc4 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:49.219685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a2e9f-9ee4-41f1-88b6-7016d4cd4bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:49.220074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a2e9f-9ee4-41f1-88b6-7016d4cd4bc4 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/46e5ca80b7834ab7bc951b454c9aae49/roles/c255d4a547bc4efd818a3ecfcd75843f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:49.222662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a2e9f-9ee4-41f1-88b6-7016d4cd4bc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=46e5ca80b7834ab7bc951b454c9aae49, role_id=c255d4a547bc4efd818a3ecfcd75843f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:49.224304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a2e9f-9ee4-41f1-88b6-7016d4cd4bc4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:49.245506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 533/1081] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:41:49 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/46e5ca80b7834ab7bc951b454c9aae49/roles/c255d4a547bc4efd818a3ecfcd75843f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:41:49.258908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:49.314845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5be17-9747-460a-b025-b06ccd393a38 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:49.361117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fddf24-e9e4-4359-8db9-76627c194947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:49.361875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fddf24-e9e4-4359-8db9-76627c194947 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:49.362155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fddf24-e9e4-4359-8db9-76627c194947 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:49.362431 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fddf24-e9e4-4359-8db9-76627c194947 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:49.445086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fddf24-e9e4-4359-8db9-76627c194947 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:49.447323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fddf24-e9e4-4359-8db9-76627c194947 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:49.509514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:49.558538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 549/1082] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:49.564479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a2e9f-9ee4-41f1-88b6-7016d4cd4bc4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:49.579092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:49.691405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a2e9f-9ee4-41f1-88b6-7016d4cd4bc4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:49.740568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a58de-af78-4f42-bd74-1c7c0b94caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:49.741380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a58de-af78-4f42-bd74-1c7c0b94caab None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:49.741662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a58de-af78-4f42-bd74-1c7c0b94caab None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:49.741981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a58de-af78-4f42-bd74-1c7c0b94caab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:49.753072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a58de-af78-4f42-bd74-1c7c0b94caab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:49.754670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a58de-af78-4f42-bd74-1c7c0b94caab None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-103a58de-af78-4f42-bd74-1c7c0b94caab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:49.774272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:49.775285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:49.781535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:49.781535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:49.781535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:49.781535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:49.781535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 534/1083] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:49.783534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fddf24-e9e4-4359-8db9-76627c194947 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:49.899093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fddf24-e9e4-4359-8db9-76627c194947 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:49.934937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc62052-92f0-43a4-9f45-554f72b27d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:49.935618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc62052-92f0-43a4-9f45-554f72b27d39 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:49.935900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc62052-92f0-43a4-9f45-554f72b27d39 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:49.936175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc62052-92f0-43a4-9f45-554f72b27d39 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:49.936917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc62052-92f0-43a4-9f45-554f72b27d39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:49.937220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc62052-92f0-43a4-9f45-554f72b27d39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:49.938649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc62052-92f0-43a4-9f45-554f72b27d39 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:49.956756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 550/1084] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:50.731344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae88ff6-2ede-44c0-8543-b29e42547ee6 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:50.732001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae88ff6-2ede-44c0-8543-b29e42547ee6 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:50.732001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae88ff6-2ede-44c0-8543-b29e42547ee6 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:50.733359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 535/1085] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:50.739261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073206af-c930-4a31-95fe-2d5166b2e036 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:50.740140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073206af-c930-4a31-95fe-2d5166b2e036 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:50.740651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073206af-c930-4a31-95fe-2d5166b2e036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:50.759046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-073206af-c930-4a31-95fe-2d5166b2e036 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:50.760236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-073206af-c930-4a31-95fe-2d5166b2e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:50.774676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:50.848473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 551/1086] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:50.872822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae88ff6-2ede-44c0-8543-b29e42547ee6 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:50.939909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:50.982764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae88ff6-2ede-44c0-8543-b29e42547ee6 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:51.017486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d0982-e0b9-4fa0-b39b-43c73a6a6f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:51.018369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d0982-e0b9-4fa0-b39b-43c73a6a6f09 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.018635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d0982-e0b9-4fa0-b39b-43c73a6a6f09 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.018905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d0982-e0b9-4fa0-b39b-43c73a6a6f09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.029589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d0982-e0b9-4fa0-b39b-43c73a6a6f09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-543d0982-e0b9-4fa0-b39b-43c73a6a6f09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:51.037529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:51.038695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:51.039635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 536/1087] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:51.045092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073206af-c930-4a31-95fe-2d5166b2e036 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:51.148589 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073206af-c930-4a31-95fe-2d5166b2e036 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:51.183329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a8fc6-7048-4e80-901c-9d3b0a5cb88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:51.184011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a8fc6-7048-4e80-901c-9d3b0a5cb88c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.184313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a8fc6-7048-4e80-901c-9d3b0a5cb88c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.184559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a8fc6-7048-4e80-901c-9d3b0a5cb88c None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.185390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a8fc6-7048-4e80-901c-9d3b0a5cb88c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:51.185700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a8fc6-7048-4e80-901c-9d3b0a5cb88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:51.187143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a8fc6-7048-4e80-901c-9d3b0a5cb88c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:51.195986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 552/1088] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:51.201920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d0982-e0b9-4fa0-b39b-43c73a6a6f09 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:51.308821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d0982-e0b9-4fa0-b39b-43c73a6a6f09 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:51.330574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e585db4-b472-4c43-b4d7-05a6c8493e59 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.330891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e585db4-b472-4c43-b4d7-05a6c8493e59 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.331176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e585db4-b472-4c43-b4d7-05a6c8493e59 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.332980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 553/1089] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:41:51.333989 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:51.339949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174088b1-8d69-41e7-870c-b5838217f203 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.340331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174088b1-8d69-41e7-870c-b5838217f203 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.340702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174088b1-8d69-41e7-870c-b5838217f203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.344953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1ffd-0fac-4afa-a30d-ad61d049c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:51.345848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1ffd-0fac-4afa-a30d-ad61d049c3bd None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.346129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1ffd-0fac-4afa-a30d-ad61d049c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.346398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf1ffd-0fac-4afa-a30d-ad61d049c3bd None admin] PATH_INFO: `/v3/users/86b90e32fb894a749f21ed4fe3b633a6` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.347670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf1ffd-0fac-4afa-a30d-ad61d049c3bd None admin] RBAC: Authorizing `identity:get_user(user_id=86b90e32fb894a749f21ed4fe3b633a6)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:51.349424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf1ffd-0fac-4afa-a30d-ad61d049c3bd None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:51.351126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 537/1090] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:41:51 2022] GET /identity/v3/users/86b90e32fb894a749f21ed4fe3b633a6 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:51.356757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1ffd-0fac-4afa-a30d-ad61d049c3bd None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:51.363966 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-174088b1-8d69-41e7-870c-b5838217f203 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:51.365178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-174088b1-8d69-41e7-870c-b5838217f203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:51.369141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:51.461758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 554/1091] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:41:51.487585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf1ffd-0fac-4afa-a30d-ad61d049c3bd None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:51.488161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174088b1-8d69-41e7-870c-b5838217f203 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:51.529881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e9fb5-e4a6-40d7-a3d6-43ae47069d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:51.530454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e9fb5-e4a6-40d7-a3d6-43ae47069d6d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.530730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e9fb5-e4a6-40d7-a3d6-43ae47069d6d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.531002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e9fb5-e4a6-40d7-a3d6-43ae47069d6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.600287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:51.606935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174088b1-8d69-41e7-870c-b5838217f203 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:51.614750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e9fb5-e4a6-40d7-a3d6-43ae47069d6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:51.616251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e9fb5-e4a6-40d7-a3d6-43ae47069d6d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:51.646555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503ae5e-3ab9-454c-a133-c33915befb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:51.647387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503ae5e-3ab9-454c-a133-c33915befb7a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.647672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503ae5e-3ab9-454c-a133-c33915befb7a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.647967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503ae5e-3ab9-454c-a133-c33915befb7a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.656445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:51.660943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503ae5e-3ab9-454c-a133-c33915befb7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-f503ae5e-3ab9-454c-a133-c33915befb7a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:41:51.670466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:51.671442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:51.672341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 555/1092] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Jun 21 15:41:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:51.678479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503ae5e-3ab9-454c-a133-c33915befb7a None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:51.724908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 538/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:51.730947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e9fb5-e4a6-40d7-a3d6-43ae47069d6d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:51.800218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503ae5e-3ab9-454c-a133-c33915befb7a None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:51.838127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:51.838835 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.839150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.839447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.839933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:51.840455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:51.840980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:51.842576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:51.852379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 556/1094] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jun 21 15:41:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:51.852783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e9fb5-e4a6-40d7-a3d6-43ae47069d6d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:51.858620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:51.892501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebd1a4-d35f-43c1-ae05-1a52ec57d728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:51.893361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebd1a4-d35f-43c1-ae05-1a52ec57d728 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:51.893636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebd1a4-d35f-43c1-ae05-1a52ec57d728 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:51.893943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebd1a4-d35f-43c1-ae05-1a52ec57d728 None admin] PATH_INFO: `/v3/projects/70b433a900fd4fc3a35b39031d787992` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:51.895399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebd1a4-d35f-43c1-ae05-1a52ec57d728 None admin] RBAC: Authorizing `identity:get_project(project_id=70b433a900fd4fc3a35b39031d787992)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:51.896832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebd1a4-d35f-43c1-ae05-1a52ec57d728 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:51.898562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 539/1095] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:41:51 2022] GET /identity/v3/projects/70b433a900fd4fc3a35b39031d787992 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:51.905304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebd1a4-d35f-43c1-ae05-1a52ec57d728 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:51.925417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:51.966908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e6ed-98d1-4124-b45d-2ccfcdb3eca3 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:52.001623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:52.002494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:52.002777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:52.003054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:52.014104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:52.014487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebd1a4-d35f-43c1-ae05-1a52ec57d728 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:52.015554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:52.033591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:52.034652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:52.035862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:52.035862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:52.035862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:52.035862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:52.035862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 557/1096] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:52.041120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:52.050933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:52.051559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:52.051846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:52.052121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:52.052964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=70b433a900fd4fc3a35b39031d787992, user.id=86b90e32fb894a749f21ed4fe3b633a6) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:52.053272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:52.054853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:52.060987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 540/1097] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:51 2022] GET /identity/v3/role_assignments?user.id=86b90e32fb894a749f21ed4fe3b633a6&scope.project.id=70b433a900fd4fc3a35b39031d787992&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:52.161387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde11f8-91e8-4aa8-a7bb-08ddcf1f81bf None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:52.179806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:52.202590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:52.203283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:52.203562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:52.203830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:52.204568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:52.204880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:52.206344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:52.220092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 558/1098] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:52.226617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:52.255784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:52.335656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61210a21-e528-4b52-86f3-590480939f90 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:52.372434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:52.373255 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:52.373535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:52.373869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:52.381912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:52.383397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:41:52.399318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:52.400334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:41:52.401290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 541/1099] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Jun 21 15:41:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:52.406660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:52.515975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4293f14-4603-42fb-87fc-c2d14f5a8420 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:52.550686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:52.551313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:52.551572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:52.551845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:52.552719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:52.553032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:52.554543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:52.572500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 559/1100] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Jun 21 15:41:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:52.578573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:52.668347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:52.687304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029321b7-defd-46de-8186-971147ffc5f4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:52.720641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ea25f-d441-45f2-9fe7-1d79599eec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:52.721301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ea25f-d441-45f2-9fe7-1d79599eec2d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:52.721576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ea25f-d441-45f2-9fe7-1d79599eec2d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:52.721880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ea25f-d441-45f2-9fe7-1d79599eec2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:52.792217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ea25f-d441-45f2-9fe7-1d79599eec2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:52.793680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ea25f-d441-45f2-9fe7-1d79599eec2d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:52.873798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:52.893232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 542/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:52.899551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:52.933264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:53.007271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4330c9b-eb74-4765-96ef-ce54229ee164 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:53.045270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:53.046095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:53.046389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:53.046659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:53.057875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:53.059341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:41:53.078479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:53.079445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:53.080323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:41:53.080323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:41:53.080323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:41:53.080323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:53.080323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 560/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:41:53.086476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ea25f-d441-45f2-9fe7-1d79599eec2d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:53.205112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ea25f-d441-45f2-9fe7-1d79599eec2d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:53.262494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:53.263191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:53.263475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:53.263755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:53.264490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:53.264799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:53.266238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:53.279461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 543/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:53.285569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:53.381405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:53.400414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c04a0-df9d-4775-a559-07a0b6ffc96c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:53.435686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:53.436539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:53.436824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:53.437092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:53.444607 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:53.446180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:41:53.460819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:41:53.461765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:41:53.462841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 561/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:41:53.468241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:53.517556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:53.591429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca147f0a-a14a-4813-a013-d6d7d395019a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:53.629189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0045b-2493-46e6-b5c8-ac1f927cdd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:53.629966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0045b-2493-46e6-b5c8-ac1f927cdd47 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:53.630249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0045b-2493-46e6-b5c8-ac1f927cdd47 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:53.630543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0045b-2493-46e6-b5c8-ac1f927cdd47 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:53.631386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0045b-2493-46e6-b5c8-ac1f927cdd47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:53.631698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0045b-2493-46e6-b5c8-ac1f927cdd47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:53.633252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0045b-2493-46e6-b5c8-ac1f927cdd47 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:53.647727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 544/1105] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:41:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:53.654451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:53.764140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ac936-c1c5-4aea-b8e9-74bb7c9f1f79 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:53.800436 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51ff89-3195-440b-82b1-21f976ae97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:53.801160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51ff89-3195-440b-82b1-21f976ae97de None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:53.801443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51ff89-3195-440b-82b1-21f976ae97de None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:53.801738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51ff89-3195-440b-82b1-21f976ae97de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:53.802599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51ff89-3195-440b-82b1-21f976ae97de None admin] RBAC: Adding query filter params (role.id=c255d4a547bc4efd818a3ecfcd75843f, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=46e5ca80b7834ab7bc951b454c9aae49) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:53.802907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51ff89-3195-440b-82b1-21f976ae97de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:53.804366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51ff89-3195-440b-82b1-21f976ae97de None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:53.810390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 562/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:53 2022] GET /identity/v3/role_assignments?user.id=46e5ca80b7834ab7bc951b454c9aae49&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=c255d4a547bc4efd818a3ecfcd75843f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:41:54.204071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dce62b-426b-4452-b2c7-f4ea4e1817d1 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:54.204624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dce62b-426b-4452-b2c7-f4ea4e1817d1 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:54.204717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dce62b-426b-4452-b2c7-f4ea4e1817d1 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:54.206362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 545/1107] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:54.212569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6b18a-d236-41f4-96de-155184d8caf2 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:54.213059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6b18a-d236-41f4-96de-155184d8caf2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:54.213145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6b18a-d236-41f4-96de-155184d8caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:54.231987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e8c6b18a-d236-41f4-96de-155184d8caf2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:54.233212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e8c6b18a-d236-41f4-96de-155184d8caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:54.235505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:54.313715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 563/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:54.341524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dce62b-426b-4452-b2c7-f4ea4e1817d1 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:54.464328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dce62b-426b-4452-b2c7-f4ea4e1817d1 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:54.466113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:54.503805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cb93d-07d5-4240-9fc2-5d71b9dca927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:54.504615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb93d-07d5-4240-9fc2-5d71b9dca927 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:54.504893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb93d-07d5-4240-9fc2-5d71b9dca927 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:54.505175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb93d-07d5-4240-9fc2-5d71b9dca927 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:54.506396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cb93d-07d5-4240-9fc2-5d71b9dca927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:54.507847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cb93d-07d5-4240-9fc2-5d71b9dca927 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:54.509496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 546/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:41:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:54.517927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6b18a-d236-41f4-96de-155184d8caf2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:54.635696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6b18a-d236-41f4-96de-155184d8caf2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:54.676098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220d5e5-0155-4812-a263-d69757f83c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:54.676422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220d5e5-0155-4812-a263-d69757f83c53 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:54.676773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220d5e5-0155-4812-a263-d69757f83c53 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:54.677158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220d5e5-0155-4812-a263-d69757f83c53 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:54.678230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220d5e5-0155-4812-a263-d69757f83c53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:54.679747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220d5e5-0155-4812-a263-d69757f83c53 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:54.689936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:54.710106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 564/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:41:54 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:41:56.769921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb321-b9ad-4523-9904-c696c332061e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:56.770503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb321-b9ad-4523-9904-c696c332061e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:56.770503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb321-b9ad-4523-9904-c696c332061e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:56.771887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 547/1111] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:56.777615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d3900-8d74-4449-9790-325c5280fd20 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:56.778070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d3900-8d74-4449-9790-325c5280fd20 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:56.778186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d3900-8d74-4449-9790-325c5280fd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:56.796215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-042d3900-8d74-4449-9790-325c5280fd20 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:56.797398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-042d3900-8d74-4449-9790-325c5280fd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:56.865449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 565/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:41:56.890180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eb321-b9ad-4523-9904-c696c332061e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:56.998347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eb321-b9ad-4523-9904-c696c332061e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:57.035244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612a825-2e70-42ef-b556-0b844165de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:57.036028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612a825-2e70-42ef-b556-0b844165de42 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:57.036307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612a825-2e70-42ef-b556-0b844165de42 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:57.036583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612a825-2e70-42ef-b556-0b844165de42 None admin] PATH_INFO: `/v3/roles/a505b2a721044e6ea8312ef6ffdc1c9f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:57.038137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612a825-2e70-42ef-b556-0b844165de42 None admin] RBAC: Authorizing `identity:get_role(role_id=a505b2a721044e6ea8312ef6ffdc1c9f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:57.039537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612a825-2e70-42ef-b556-0b844165de42 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:57.040923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 548/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:41:56 2022] GET /identity/v3/roles/a505b2a721044e6ea8312ef6ffdc1c9f => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:57.046986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d3900-8d74-4449-9790-325c5280fd20 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:57.152670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d3900-8d74-4449-9790-325c5280fd20 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:57.176944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:57.189120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111a10d-b0cb-43f1-9b64-755b4c377f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:57.190155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111a10d-b0cb-43f1-9b64-755b4c377f2c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:57.190434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111a10d-b0cb-43f1-9b64-755b4c377f2c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:57.190777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111a10d-b0cb-43f1-9b64-755b4c377f2c None admin] PATH_INFO: `/v3/users/a99e9709ca2348db8f0f5eb48872a7bc` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:57.204876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111a10d-b0cb-43f1-9b64-755b4c377f2c None admin] RBAC: Authorizing `identity:get_user(user_id=a99e9709ca2348db8f0f5eb48872a7bc)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:57.206430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111a10d-b0cb-43f1-9b64-755b4c377f2c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:57.208225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 566/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:41:57 2022] GET /identity/v3/users/a99e9709ca2348db8f0f5eb48872a7bc => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:57.214385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612a825-2e70-42ef-b556-0b844165de42 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:57.257937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:57.321255 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612a825-2e70-42ef-b556-0b844165de42 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:57.357524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa94bf-7c26-4f2f-a5ca-36c3b6d044eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:57.358214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa94bf-7c26-4f2f-a5ca-36c3b6d044eb None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:57.358497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa94bf-7c26-4f2f-a5ca-36c3b6d044eb None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:57.358777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa94bf-7c26-4f2f-a5ca-36c3b6d044eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:57.428757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa94bf-7c26-4f2f-a5ca-36c3b6d044eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:57.430257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa94bf-7c26-4f2f-a5ca-36c3b6d044eb None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:57.470292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:57.534863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 549/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:41:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:41:57.541020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111a10d-b0cb-43f1-9b64-755b4c377f2c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:57.650830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111a10d-b0cb-43f1-9b64-755b4c377f2c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:57.689525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a3acb-c10c-4d78-97a1-7a2959f2b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:57.690423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a3acb-c10c-4d78-97a1-7a2959f2b05a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:57.690745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a3acb-c10c-4d78-97a1-7a2959f2b05a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:57.691021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a3acb-c10c-4d78-97a1-7a2959f2b05a None admin] PATH_INFO: `/v3/projects/70b433a900fd4fc3a35b39031d787992` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:57.692240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a3acb-c10c-4d78-97a1-7a2959f2b05a None admin] RBAC: Authorizing `identity:get_project(project_id=70b433a900fd4fc3a35b39031d787992)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:57.693676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a3acb-c10c-4d78-97a1-7a2959f2b05a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:57.695526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 567/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:41:57 2022] GET /identity/v3/projects/70b433a900fd4fc3a35b39031d787992 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:57.702032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa94bf-7c26-4f2f-a5ca-36c3b6d044eb None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:41:57.807782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa94bf-7c26-4f2f-a5ca-36c3b6d044eb None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:41:57.844301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf838a3-535e-40b0-a108-d4d1f980e64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:41:57.844984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf838a3-535e-40b0-a108-d4d1f980e64f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:57.845272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf838a3-535e-40b0-a108-d4d1f980e64f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:57.845555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf838a3-535e-40b0-a108-d4d1f980e64f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:57.846468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf838a3-535e-40b0-a108-d4d1f980e64f None admin] RBAC: Adding query filter params (role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=70b433a900fd4fc3a35b39031d787992, user.id=a99e9709ca2348db8f0f5eb48872a7bc) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:41:57.846788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf838a3-535e-40b0-a108-d4d1f980e64f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:41:57.848527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf838a3-535e-40b0-a108-d4d1f980e64f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:41:57.854262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 550/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:41:57 2022] GET /identity/v3/role_assignments?user.id=a99e9709ca2348db8f0f5eb48872a7bc&scope.project.id=70b433a900fd4fc3a35b39031d787992&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:41:59.883673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd773f9-053b-4d9c-869b-b6171c340d57 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:59.884230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd773f9-053b-4d9c-869b-b6171c340d57 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:59.884230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd773f9-053b-4d9c-869b-b6171c340d57 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:59.885626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 568/1118] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:41:59.891370 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37fb3-5c51-49ec-a45d-d930eaecef69 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:41:59.891819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37fb3-5c51-49ec-a45d-d930eaecef69 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:41:59.891911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37fb3-5c51-49ec-a45d-d930eaecef69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:41:59.897347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:41:59.910523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1bb37fb3-5c51-49ec-a45d-d930eaecef69 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:41:59.911724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1bb37fb3-5c51-49ec-a45d-d930eaecef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:41:59.981229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 551/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:41:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:00.005137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd773f9-053b-4d9c-869b-b6171c340d57 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:00.056903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:00.107621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd773f9-053b-4d9c-869b-b6171c340d57 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:00.143335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfada915-5cc2-4eaf-b557-9a696c7d5b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:00.144153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfada915-5cc2-4eaf-b557-9a696c7d5b48 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:00.144430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfada915-5cc2-4eaf-b557-9a696c7d5b48 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:00.144700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfada915-5cc2-4eaf-b557-9a696c7d5b48 None admin] PATH_INFO: `/v3/roles/a505b2a721044e6ea8312ef6ffdc1c9f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:00.146325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfada915-5cc2-4eaf-b557-9a696c7d5b48 None admin] RBAC: Authorizing `identity:get_role(role_id=a505b2a721044e6ea8312ef6ffdc1c9f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:00.147736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfada915-5cc2-4eaf-b557-9a696c7d5b48 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:00.149397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 569/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:00 2022] GET /identity/v3/roles/a505b2a721044e6ea8312ef6ffdc1c9f => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:00.156286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37fb3-5c51-49ec-a45d-d930eaecef69 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:00.258977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37fb3-5c51-49ec-a45d-d930eaecef69 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:00.290394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:00.296824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae888e-786e-4044-b859-8df6a9e990e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:00.297998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae888e-786e-4044-b859-8df6a9e990e4 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:00.298292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae888e-786e-4044-b859-8df6a9e990e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:00.298569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae888e-786e-4044-b859-8df6a9e990e4 None admin] PATH_INFO: `/v3/users/a99e9709ca2348db8f0f5eb48872a7bc` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:00.300074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae888e-786e-4044-b859-8df6a9e990e4 None admin] RBAC: Authorizing `identity:get_user(user_id=a99e9709ca2348db8f0f5eb48872a7bc)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:00.301802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae888e-786e-4044-b859-8df6a9e990e4 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:00.303839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 552/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:00 2022] GET /identity/v3/users/a99e9709ca2348db8f0f5eb48872a7bc => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:00.310444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfada915-5cc2-4eaf-b557-9a696c7d5b48 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:00.421921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfada915-5cc2-4eaf-b557-9a696c7d5b48 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:00.441454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:00.461491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25177c-28f6-4348-b7f2-11f9d3f1b172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:00.462410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25177c-28f6-4348-b7f2-11f9d3f1b172 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:00.462696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25177c-28f6-4348-b7f2-11f9d3f1b172 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:00.462975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25177c-28f6-4348-b7f2-11f9d3f1b172 None admin] PATH_INFO: `/v3/projects/70b433a900fd4fc3a35b39031d787992` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:00.464179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25177c-28f6-4348-b7f2-11f9d3f1b172 None admin] RBAC: Authorizing `identity:get_project(project_id=70b433a900fd4fc3a35b39031d787992)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:00.465637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25177c-28f6-4348-b7f2-11f9d3f1b172 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:00.467417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 570/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:42:00 2022] GET /identity/v3/projects/70b433a900fd4fc3a35b39031d787992 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:00.473754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae888e-786e-4044-b859-8df6a9e990e4 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:00.578553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae888e-786e-4044-b859-8df6a9e990e4 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:00.614455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7fbdd-e9b8-49ff-bdcd-9f74b502ee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:00.615380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fbdd-e9b8-49ff-bdcd-9f74b502ee5e None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:00.615668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fbdd-e9b8-49ff-bdcd-9f74b502ee5e None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:00.615941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7fbdd-e9b8-49ff-bdcd-9f74b502ee5e None admin] PATH_INFO: `/v3/projects/70b433a900fd4fc3a35b39031d787992/users/a99e9709ca2348db8f0f5eb48872a7bc/roles/a505b2a721044e6ea8312ef6ffdc1c9f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:00.618315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7fbdd-e9b8-49ff-bdcd-9f74b502ee5e None admin] RBAC: Authorizing `identity:create_grant(project_id=70b433a900fd4fc3a35b39031d787992, user_id=a99e9709ca2348db8f0f5eb48872a7bc, role_id=a505b2a721044e6ea8312ef6ffdc1c9f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:00.619758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7fbdd-e9b8-49ff-bdcd-9f74b502ee5e None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:00.634526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 553/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:42:00 2022] PUT /identity/v3/projects/70b433a900fd4fc3a35b39031d787992/users/a99e9709ca2348db8f0f5eb48872a7bc/roles/a505b2a721044e6ea8312ef6ffdc1c9f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:42:02.645810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6e95b-fa42-4222-92a2-72976cd5aa79 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:02.646470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6e95b-fa42-4222-92a2-72976cd5aa79 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:02.646470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6e95b-fa42-4222-92a2-72976cd5aa79 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:02.647767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 571/1124] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:02.653648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e76c0-b856-4829-a3f8-27c49b61c69e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:02.654078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e76c0-b856-4829-a3f8-27c49b61c69e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:02.654078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e76c0-b856-4829-a3f8-27c49b61c69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:02.672601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a86e76c0-b856-4829-a3f8-27c49b61c69e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:02.673881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a86e76c0-b856-4829-a3f8-27c49b61c69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:02.733264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:02.747462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 554/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:02.772277 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6e95b-fa42-4222-92a2-72976cd5aa79 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:02.890643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6e95b-fa42-4222-92a2-72976cd5aa79 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:02.925866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc372d9-c2ff-42a2-8876-0b4957c8c284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:02.926667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc372d9-c2ff-42a2-8876-0b4957c8c284 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:02.926945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc372d9-c2ff-42a2-8876-0b4957c8c284 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:02.927208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc372d9-c2ff-42a2-8876-0b4957c8c284 None admin] PATH_INFO: `/v3/roles/a505b2a721044e6ea8312ef6ffdc1c9f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:02.928678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc372d9-c2ff-42a2-8876-0b4957c8c284 None admin] RBAC: Authorizing `identity:get_role(role_id=a505b2a721044e6ea8312ef6ffdc1c9f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:02.930089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc372d9-c2ff-42a2-8876-0b4957c8c284 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:02.931460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 572/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:02 2022] GET /identity/v3/roles/a505b2a721044e6ea8312ef6ffdc1c9f => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:02.937497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e76c0-b856-4829-a3f8-27c49b61c69e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:03.042670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e76c0-b856-4829-a3f8-27c49b61c69e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:03.078193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c7e55-ae0f-4d49-b59f-d2d8643d1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:03.079067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7e55-ae0f-4d49-b59f-d2d8643d1656 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:03.079344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7e55-ae0f-4d49-b59f-d2d8643d1656 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:03.079615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7e55-ae0f-4d49-b59f-d2d8643d1656 None admin] PATH_INFO: `/v3/users/a99e9709ca2348db8f0f5eb48872a7bc` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:03.080885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c7e55-ae0f-4d49-b59f-d2d8643d1656 None admin] RBAC: Authorizing `identity:get_user(user_id=a99e9709ca2348db8f0f5eb48872a7bc)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:03.082392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c7e55-ae0f-4d49-b59f-d2d8643d1656 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:03.084031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 555/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:02 2022] GET /identity/v3/users/a99e9709ca2348db8f0f5eb48872a7bc => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:03.090173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc372d9-c2ff-42a2-8876-0b4957c8c284 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:03.158303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:03.204726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc372d9-c2ff-42a2-8876-0b4957c8c284 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:03.244493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89321598-d5dc-4c15-ae8c-e55ce93c97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:03.245198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89321598-d5dc-4c15-ae8c-e55ce93c97c4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:03.245480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89321598-d5dc-4c15-ae8c-e55ce93c97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:03.245750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89321598-d5dc-4c15-ae8c-e55ce93c97c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:03.314904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89321598-d5dc-4c15-ae8c-e55ce93c97c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:03.316402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89321598-d5dc-4c15-ae8c-e55ce93c97c4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:03.372673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:03.421785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 573/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:42:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:42:03.428319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c7e55-ae0f-4d49-b59f-d2d8643d1656 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:03.495576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:03.541221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c7e55-ae0f-4d49-b59f-d2d8643d1656 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:03.576757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15d062-4920-4e9c-a929-74baa1b558b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:03.577745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15d062-4920-4e9c-a929-74baa1b558b2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:03.578074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15d062-4920-4e9c-a929-74baa1b558b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:03.578425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15d062-4920-4e9c-a929-74baa1b558b2 None admin] PATH_INFO: `/v3/projects/70b433a900fd4fc3a35b39031d787992` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:03.579667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15d062-4920-4e9c-a929-74baa1b558b2 None admin] RBAC: Authorizing `identity:get_project(project_id=70b433a900fd4fc3a35b39031d787992)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:03.581808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15d062-4920-4e9c-a929-74baa1b558b2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:03.583511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 556/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:42:03 2022] GET /identity/v3/projects/70b433a900fd4fc3a35b39031d787992 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:03.590343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89321598-d5dc-4c15-ae8c-e55ce93c97c4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:03.695094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89321598-d5dc-4c15-ae8c-e55ce93c97c4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:03.731726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f55f96-8b0c-4599-b552-b102d3a23503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:03.732378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f55f96-8b0c-4599-b552-b102d3a23503 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:03.732657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f55f96-8b0c-4599-b552-b102d3a23503 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:03.732929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f55f96-8b0c-4599-b552-b102d3a23503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:03.733758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f55f96-8b0c-4599-b552-b102d3a23503 None admin] RBAC: Adding query filter params (role.id=a505b2a721044e6ea8312ef6ffdc1c9f, scope.project.id=70b433a900fd4fc3a35b39031d787992, user.id=a99e9709ca2348db8f0f5eb48872a7bc) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:03.734098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f55f96-8b0c-4599-b552-b102d3a23503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:03.735525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f55f96-8b0c-4599-b552-b102d3a23503 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:03.741671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 574/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:42:03 2022] GET /identity/v3/role_assignments?user.id=a99e9709ca2348db8f0f5eb48872a7bc&scope.project.id=70b433a900fd4fc3a35b39031d787992&role.id=a505b2a721044e6ea8312ef6ffdc1c9f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:05.767810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f2180-2f9a-4b24-a1b5-da3f03ac3a61 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:05.768395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f2180-2f9a-4b24-a1b5-da3f03ac3a61 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:05.768602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f2180-2f9a-4b24-a1b5-da3f03ac3a61 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:05.770000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 557/1131] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:05.775501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0e49-63f5-4675-8d18-44006de9da3c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:05.775911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0e49-63f5-4675-8d18-44006de9da3c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:05.776057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0e49-63f5-4675-8d18-44006de9da3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:05.796516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c43d0e49-63f5-4675-8d18-44006de9da3c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:05.798305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c43d0e49-63f5-4675-8d18-44006de9da3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:05.814486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:05.874481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 575/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:05.899187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f2180-2f9a-4b24-a1b5-da3f03ac3a61 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:06.006532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f2180-2f9a-4b24-a1b5-da3f03ac3a61 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:06.032953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:06.042076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6f155-d915-4c5a-80f8-594c380b296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:06.042880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6f155-d915-4c5a-80f8-594c380b296a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:06.043159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6f155-d915-4c5a-80f8-594c380b296a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:06.043435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6f155-d915-4c5a-80f8-594c380b296a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:06.044641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6f155-d915-4c5a-80f8-594c380b296a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:06.046110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6f155-d915-4c5a-80f8-594c380b296a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:06.047728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 558/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:42:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:06.054117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d0e49-63f5-4675-8d18-44006de9da3c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:06.161688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d0e49-63f5-4675-8d18-44006de9da3c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:06.199072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfc995-331e-4d06-9f94-aaa789d73434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:06.199771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfc995-331e-4d06-9f94-aaa789d73434 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:06.200054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfc995-331e-4d06-9f94-aaa789d73434 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:06.200332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfc995-331e-4d06-9f94-aaa789d73434 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:06.201046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfc995-331e-4d06-9f94-aaa789d73434 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:06.202566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfc995-331e-4d06-9f94-aaa789d73434 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:06.224457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:06.232357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 576/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:42:06 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:42:08.328296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb214-0f58-4d4f-99f8-a70724d621b8 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:08.328892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb214-0f58-4d4f-99f8-a70724d621b8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:08.328892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb214-0f58-4d4f-99f8-a70724d621b8 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:08.330280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 559/1135] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:08.337934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1c27-9836-4898-80db-0e8b95a46a54 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:08.338475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1c27-9836-4898-80db-0e8b95a46a54 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:08.338845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1c27-9836-4898-80db-0e8b95a46a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:08.359805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-cd7a1c27-9836-4898-80db-0e8b95a46a54 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:08.360986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cd7a1c27-9836-4898-80db-0e8b95a46a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:08.435698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 577/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:08.461654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fb214-0f58-4d4f-99f8-a70724d621b8 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:08.584107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fb214-0f58-4d4f-99f8-a70724d621b8 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:08.646044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d74ba-5a14-4189-ae54-e129eb827f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:08.646873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d74ba-5a14-4189-ae54-e129eb827f78 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:08.647150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d74ba-5a14-4189-ae54-e129eb827f78 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:08.647427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d74ba-5a14-4189-ae54-e129eb827f78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:08.648702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d74ba-5a14-4189-ae54-e129eb827f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:08.650224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d74ba-5a14-4189-ae54-e129eb827f78 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:08.651884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 560/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:42:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:08.658348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a1c27-9836-4898-80db-0e8b95a46a54 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:08.762556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a1c27-9836-4898-80db-0e8b95a46a54 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:08.796169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:08.803243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1248e-c56b-4646-b176-f16e211ca377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:08.803927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1248e-c56b-4646-b176-f16e211ca377 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:08.804209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1248e-c56b-4646-b176-f16e211ca377 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:08.804483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1248e-c56b-4646-b176-f16e211ca377 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:08.805389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1248e-c56b-4646-b176-f16e211ca377 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:08.806835 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1248e-c56b-4646-b176-f16e211ca377 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:08.828206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 578/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:42:08 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:42:10.912159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f4540-0fc7-4faf-b951-4ddda3b7e7ec None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:10.912778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f4540-0fc7-4faf-b951-4ddda3b7e7ec None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:10.912778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f4540-0fc7-4faf-b951-4ddda3b7e7ec None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:10.914152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 561/1139] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:10.919862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcec70-36f2-4f91-81c6-a6709d73a96e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:10.920401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcec70-36f2-4f91-81c6-a6709d73a96e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:10.920502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddcec70-36f2-4f91-81c6-a6709d73a96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:10.938550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-9ddcec70-36f2-4f91-81c6-a6709d73a96e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:10.939978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9ddcec70-36f2-4f91-81c6-a6709d73a96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:11.007464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 579/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:11.032248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f4540-0fc7-4faf-b951-4ddda3b7e7ec None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:11.103904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:11.145813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f4540-0fc7-4faf-b951-4ddda3b7e7ec None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:11.182121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ae2c3-87dd-4477-82b2-50fc6c8f4889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:11.182846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ae2c3-87dd-4477-82b2-50fc6c8f4889 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:11.183136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ae2c3-87dd-4477-82b2-50fc6c8f4889 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:11.183407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ae2c3-87dd-4477-82b2-50fc6c8f4889 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:11.194223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ae2c3-87dd-4477-82b2-50fc6c8f4889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-594ae2c3-87dd-4477-82b2-50fc6c8f4889 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:42:11.201899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:42:11.202820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:42:11.203736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 562/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:42:11.209998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcec70-36f2-4f91-81c6-a6709d73a96e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:11.315776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddcec70-36f2-4f91-81c6-a6709d73a96e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:11.334789 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:11.354040 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:11.354722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:11.355006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:11.355284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:11.356063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:11.356371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:11.357816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:11.366560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 580/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:11.372582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ae2c3-87dd-4477-82b2-50fc6c8f4889 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:11.479910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ae2c3-87dd-4477-82b2-50fc6c8f4889 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:11.502947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:11.516084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96c37-80c5-4d9e-8416-f211045f7786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:11.516963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96c37-80c5-4d9e-8416-f211045f7786 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:11.517246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96c37-80c5-4d9e-8416-f211045f7786 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:11.517520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd96c37-80c5-4d9e-8416-f211045f7786 None admin] PATH_INFO: `/v3/users/1c8ce42da94b4ffb9dae57018db7690a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:11.532611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd96c37-80c5-4d9e-8416-f211045f7786 None admin] RBAC: Authorizing `identity:get_user(user_id=1c8ce42da94b4ffb9dae57018db7690a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:11.534124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd96c37-80c5-4d9e-8416-f211045f7786 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:11.535857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 563/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:11 2022] GET /identity/v3/users/1c8ce42da94b4ffb9dae57018db7690a => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:11.541842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:11.648258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4291a-91c4-466e-adfa-278649f24048 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:11.684145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3f3cc-8a38-43dd-bc79-5b5a1a7555a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:11.684839 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f3cc-8a38-43dd-bc79-5b5a1a7555a1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:11.685116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f3cc-8a38-43dd-bc79-5b5a1a7555a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:11.685391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f3cc-8a38-43dd-bc79-5b5a1a7555a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:11.731582 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:11.756382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3f3cc-8a38-43dd-bc79-5b5a1a7555a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:11.757925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3f3cc-8a38-43dd-bc79-5b5a1a7555a1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:11.859544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 581/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:42:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:42:11.866206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96c37-80c5-4d9e-8416-f211045f7786 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:11.989171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd96c37-80c5-4d9e-8416-f211045f7786 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:12.028187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958292f-4447-4b5a-8fa9-b44ea3b08153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:12.029053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958292f-4447-4b5a-8fa9-b44ea3b08153 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:12.029332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958292f-4447-4b5a-8fa9-b44ea3b08153 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:12.029603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958292f-4447-4b5a-8fa9-b44ea3b08153 None admin] PATH_INFO: `/v3/projects/7588f27ca2454eecbccc30b6d50797bc` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:12.030877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958292f-4447-4b5a-8fa9-b44ea3b08153 None admin] RBAC: Authorizing `identity:get_project(project_id=7588f27ca2454eecbccc30b6d50797bc)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:12.032310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958292f-4447-4b5a-8fa9-b44ea3b08153 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:12.034093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 564/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:42:11 2022] GET /identity/v3/projects/7588f27ca2454eecbccc30b6d50797bc => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:12.040693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3f3cc-8a38-43dd-bc79-5b5a1a7555a1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:12.124029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:12.148969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3f3cc-8a38-43dd-bc79-5b5a1a7555a1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:12.184921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715b638-afdf-4b5b-8948-bc4787c12c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:12.185569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715b638-afdf-4b5b-8948-bc4787c12c78 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:12.185891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715b638-afdf-4b5b-8948-bc4787c12c78 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:12.186166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715b638-afdf-4b5b-8948-bc4787c12c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:12.187005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715b638-afdf-4b5b-8948-bc4787c12c78 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=7588f27ca2454eecbccc30b6d50797bc, user.id=1c8ce42da94b4ffb9dae57018db7690a) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:12.187315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715b638-afdf-4b5b-8948-bc4787c12c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:12.188769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715b638-afdf-4b5b-8948-bc4787c12c78 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:12.195609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 582/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:42:12 2022] GET /identity/v3/role_assignments?user.id=1c8ce42da94b4ffb9dae57018db7690a&scope.project.id=7588f27ca2454eecbccc30b6d50797bc&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:14.268645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92c276-8358-4f15-a499-90c630d74319 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:14.269167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92c276-8358-4f15-a499-90c630d74319 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:14.269167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92c276-8358-4f15-a499-90c630d74319 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:14.270600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 565/1147] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:14.276677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85578a-4b15-4f5d-9bde-a2ce1a4cde4d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:14.277502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85578a-4b15-4f5d-9bde-a2ce1a4cde4d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:14.277502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85578a-4b15-4f5d-9bde-a2ce1a4cde4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:14.302439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5b85578a-4b15-4f5d-9bde-a2ce1a4cde4d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:14.303627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5b85578a-4b15-4f5d-9bde-a2ce1a4cde4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:14.376275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 583/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:14.402205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92c276-8358-4f15-a499-90c630d74319 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:14.475882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:14.539102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92c276-8358-4f15-a499-90c630d74319 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:14.576712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff1ae8-813c-4f79-9142-cf6c0e4d0e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:14.577505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff1ae8-813c-4f79-9142-cf6c0e4d0e69 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:14.577818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff1ae8-813c-4f79-9142-cf6c0e4d0e69 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:14.578142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff1ae8-813c-4f79-9142-cf6c0e4d0e69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:14.590408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff1ae8-813c-4f79-9142-cf6c0e4d0e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-feff1ae8-813c-4f79-9142-cf6c0e4d0e69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:42:14.598627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:42:14.599709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:42:14.600726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 566/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:42:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:42:14.607256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85578a-4b15-4f5d-9bde-a2ce1a4cde4d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:14.680577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:14.712680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85578a-4b15-4f5d-9bde-a2ce1a4cde4d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:14.756429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:14.757101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:14.757384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:14.757659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:14.758442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:14.758749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:14.760388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:14.773316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 584/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:42:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:14.779247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff1ae8-813c-4f79-9142-cf6c0e4d0e69 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:14.884790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff1ae8-813c-4f79-9142-cf6c0e4d0e69 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:14.894665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:14.922917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70450fb-e7d5-42bf-a2d0-9d3624d0b8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:14.923823 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70450fb-e7d5-42bf-a2d0-9d3624d0b8a5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:14.924110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70450fb-e7d5-42bf-a2d0-9d3624d0b8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:14.924385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70450fb-e7d5-42bf-a2d0-9d3624d0b8a5 None admin] PATH_INFO: `/v3/users/1c8ce42da94b4ffb9dae57018db7690a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:14.925617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70450fb-e7d5-42bf-a2d0-9d3624d0b8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=1c8ce42da94b4ffb9dae57018db7690a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:14.927091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70450fb-e7d5-42bf-a2d0-9d3624d0b8a5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:14.928781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 567/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:14 2022] GET /identity/v3/users/1c8ce42da94b4ffb9dae57018db7690a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:14.934831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:15.046430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3223-957c-46bd-b610-25b5e547eaa6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:15.084575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa129f4-f531-416f-b998-2c5b01d366a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:15.085451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa129f4-f531-416f-b998-2c5b01d366a0 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:15.085730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa129f4-f531-416f-b998-2c5b01d366a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:15.086039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa129f4-f531-416f-b998-2c5b01d366a0 None admin] PATH_INFO: `/v3/projects/7588f27ca2454eecbccc30b6d50797bc` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:15.087365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa129f4-f531-416f-b998-2c5b01d366a0 None admin] RBAC: Authorizing `identity:get_project(project_id=7588f27ca2454eecbccc30b6d50797bc)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:15.088831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa129f4-f531-416f-b998-2c5b01d366a0 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:15.090745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 585/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:42:14 2022] GET /identity/v3/projects/7588f27ca2454eecbccc30b6d50797bc => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:15.097550 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70450fb-e7d5-42bf-a2d0-9d3624d0b8a5 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:15.203222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70450fb-e7d5-42bf-a2d0-9d3624d0b8a5 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:15.239296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598efb5-112f-439d-b682-b6bc7a265c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:15.240465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598efb5-112f-439d-b682-b6bc7a265c0b None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:15.240827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598efb5-112f-439d-b682-b6bc7a265c0b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:15.241187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598efb5-112f-439d-b682-b6bc7a265c0b None admin] PATH_INFO: `/v3/projects/7588f27ca2454eecbccc30b6d50797bc/users/1c8ce42da94b4ffb9dae57018db7690a/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:15.244089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598efb5-112f-439d-b682-b6bc7a265c0b None admin] RBAC: Authorizing `identity:create_grant(project_id=7588f27ca2454eecbccc30b6d50797bc, user_id=1c8ce42da94b4ffb9dae57018db7690a, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:15.246239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598efb5-112f-439d-b682-b6bc7a265c0b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:15.261718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 568/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:42:15 2022] PUT /identity/v3/projects/7588f27ca2454eecbccc30b6d50797bc/users/1c8ce42da94b4ffb9dae57018db7690a/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:42:17.286244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e55126-6dfe-4f43-bc64-dc3c129bf1f5 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:17.287152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e55126-6dfe-4f43-bc64-dc3c129bf1f5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:17.287313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e55126-6dfe-4f43-bc64-dc3c129bf1f5 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:17.289213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 586/1154] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:17.298277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719dd9b-4d77-44f0-98d5-d0bf5b658342 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:17.298905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719dd9b-4d77-44f0-98d5-d0bf5b658342 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:17.299188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719dd9b-4d77-44f0-98d5-d0bf5b658342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:17.317599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1719dd9b-4d77-44f0-98d5-d0bf5b658342 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:17.318949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1719dd9b-4d77-44f0-98d5-d0bf5b658342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:17.355171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:17.398688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 569/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:17.423612 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e55126-6dfe-4f43-bc64-dc3c129bf1f5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:17.427178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:17.541683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e55126-6dfe-4f43-bc64-dc3c129bf1f5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:17.577329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f437205-fc04-49d1-9ec5-9808d2415502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:17.578156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f437205-fc04-49d1-9ec5-9808d2415502 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:17.578438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f437205-fc04-49d1-9ec5-9808d2415502 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:17.578711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f437205-fc04-49d1-9ec5-9808d2415502 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:17.589855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f437205-fc04-49d1-9ec5-9808d2415502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-2f437205-fc04-49d1-9ec5-9808d2415502 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:42:17.597805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:42:17.599173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:42:17.600196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 587/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:42:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:42:17.605819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719dd9b-4d77-44f0-98d5-d0bf5b658342 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:17.720607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719dd9b-4d77-44f0-98d5-d0bf5b658342 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:17.761863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:17.762691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:17.762979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:17.763286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:17.764085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:17.764393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:17.766308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:17.776364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 570/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:17.783195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f437205-fc04-49d1-9ec5-9808d2415502 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:17.852508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:17.896822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f437205-fc04-49d1-9ec5-9808d2415502 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:17.935985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871ea4e-4207-4700-b801-8a870dc93717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:17.936977 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871ea4e-4207-4700-b801-8a870dc93717 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:17.937291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871ea4e-4207-4700-b801-8a870dc93717 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:17.937573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871ea4e-4207-4700-b801-8a870dc93717 None admin] PATH_INFO: `/v3/users/1c8ce42da94b4ffb9dae57018db7690a` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:17.938864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871ea4e-4207-4700-b801-8a870dc93717 None admin] RBAC: Authorizing `identity:get_user(user_id=1c8ce42da94b4ffb9dae57018db7690a)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:17.940575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871ea4e-4207-4700-b801-8a870dc93717 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:17.942242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 588/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:17 2022] GET /identity/v3/users/1c8ce42da94b4ffb9dae57018db7690a => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:17.950137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:17.982769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:18.059686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c06449-72ef-4cc2-abc9-5428783ec693 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:18.097900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f87a18-402b-4bb1-8c89-e9cdb91835f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:18.098547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f87a18-402b-4bb1-8c89-e9cdb91835f3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:18.098834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f87a18-402b-4bb1-8c89-e9cdb91835f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:18.099112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f87a18-402b-4bb1-8c89-e9cdb91835f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:18.177686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f87a18-402b-4bb1-8c89-e9cdb91835f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:18.179106 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f87a18-402b-4bb1-8c89-e9cdb91835f3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:18.208760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:18.291711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 571/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:42:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:42:18.297925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871ea4e-4207-4700-b801-8a870dc93717 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:18.412655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871ea4e-4207-4700-b801-8a870dc93717 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:18.443601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:18.449521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e873f-74a8-4d24-86df-78c85a739dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:18.450463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e873f-74a8-4d24-86df-78c85a739dc8 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:18.450741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e873f-74a8-4d24-86df-78c85a739dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:18.451011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e873f-74a8-4d24-86df-78c85a739dc8 None admin] PATH_INFO: `/v3/projects/7588f27ca2454eecbccc30b6d50797bc` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:18.452218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e873f-74a8-4d24-86df-78c85a739dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=7588f27ca2454eecbccc30b6d50797bc)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:18.453736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e873f-74a8-4d24-86df-78c85a739dc8 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:18.455645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 589/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:42:18 2022] GET /identity/v3/projects/7588f27ca2454eecbccc30b6d50797bc => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:18.462479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f87a18-402b-4bb1-8c89-e9cdb91835f3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:18.570956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f87a18-402b-4bb1-8c89-e9cdb91835f3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:18.600207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:18.609076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4566c9-53d6-43ea-9bb5-995b365411f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:18.609742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4566c9-53d6-43ea-9bb5-995b365411f6 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:18.610045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4566c9-53d6-43ea-9bb5-995b365411f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:18.610319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4566c9-53d6-43ea-9bb5-995b365411f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:18.611368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4566c9-53d6-43ea-9bb5-995b365411f6 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=7588f27ca2454eecbccc30b6d50797bc, user.id=1c8ce42da94b4ffb9dae57018db7690a) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:18.611794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4566c9-53d6-43ea-9bb5-995b365411f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:18.613475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4566c9-53d6-43ea-9bb5-995b365411f6 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:18.620481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 572/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:42:18 2022] GET /identity/v3/role_assignments?user.id=1c8ce42da94b4ffb9dae57018db7690a&scope.project.id=7588f27ca2454eecbccc30b6d50797bc&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:20.686336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c963b7-0d71-4584-b847-35d4c5217ae0 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:20.686842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c963b7-0d71-4584-b847-35d4c5217ae0 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:20.686842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c963b7-0d71-4584-b847-35d4c5217ae0 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:20.688160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 590/1162] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:20.693925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4395-53bd-45d1-b3ce-89206ee964c8 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:20.694875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4395-53bd-45d1-b3ce-89206ee964c8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:20.694875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4395-53bd-45d1-b3ce-89206ee964c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:20.717500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-db8e4395-53bd-45d1-b3ce-89206ee964c8 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:20.718873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-db8e4395-53bd-45d1-b3ce-89206ee964c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:20.796437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 573/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:20.825492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c963b7-0d71-4584-b847-35d4c5217ae0 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:20.954515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c963b7-0d71-4584-b847-35d4c5217ae0 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:20.994059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e45f-275b-4fa0-a5aa-d1c6c8504881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:20.994715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e45f-275b-4fa0-a5aa-d1c6c8504881 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:20.995011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e45f-275b-4fa0-a5aa-d1c6c8504881 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:20.995318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e45f-275b-4fa0-a5aa-d1c6c8504881 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:21.071505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:21.076774 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e45f-275b-4fa0-a5aa-d1c6c8504881 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:21.078530 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e45f-275b-4fa0-a5aa-d1c6c8504881 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:21.197872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 591/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:42:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:42:21.207760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e4395-53bd-45d1-b3ce-89206ee964c8 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:21.348294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e4395-53bd-45d1-b3ce-89206ee964c8 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:21.382425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:21.397411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e007c-8149-47e7-8565-92e23a1c73c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:21.398274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e007c-8149-47e7-8565-92e23a1c73c1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:21.398558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e007c-8149-47e7-8565-92e23a1c73c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:21.398838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e007c-8149-47e7-8565-92e23a1c73c1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:21.409095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e007c-8149-47e7-8565-92e23a1c73c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:21.410603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e007c-8149-47e7-8565-92e23a1c73c1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-2a7e007c-8149-47e7-8565-92e23a1c73c1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:42:21.429001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:42:21.430991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:42:21.432084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 574/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:42:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:42:21.436735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e45f-275b-4fa0-a5aa-d1c6c8504881 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:21.561139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:21.566131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e45f-275b-4fa0-a5aa-d1c6c8504881 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:21.608596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016d953-d35d-4762-9b8d-7a8ecbcf61d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:21.609299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016d953-d35d-4762-9b8d-7a8ecbcf61d3 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:21.609618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016d953-d35d-4762-9b8d-7a8ecbcf61d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:21.609953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016d953-d35d-4762-9b8d-7a8ecbcf61d3 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:21.610746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016d953-d35d-4762-9b8d-7a8ecbcf61d3 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:21.611056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016d953-d35d-4762-9b8d-7a8ecbcf61d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:21.612502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016d953-d35d-4762-9b8d-7a8ecbcf61d3 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:21.620540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 592/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Jun 21 15:42:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:42:23.823573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cc99-d135-4c94-bba1-ff0211934801 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:23.824162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cc99-d135-4c94-bba1-ff0211934801 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:23.824162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cc99-d135-4c94-bba1-ff0211934801 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:23.825587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 575/1167] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:23.833084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188314ab-4d43-42b4-b29f-35da746a1548 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:23.833590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188314ab-4d43-42b4-b29f-35da746a1548 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:23.833590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188314ab-4d43-42b4-b29f-35da746a1548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:23.856145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-188314ab-4d43-42b4-b29f-35da746a1548 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:23.857319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-188314ab-4d43-42b4-b29f-35da746a1548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:23.929879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 593/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:42:23.956547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25cc99-d135-4c94-bba1-ff0211934801 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:24.066853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25cc99-d135-4c94-bba1-ff0211934801 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:24.102814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb1687-0ae5-45d4-97b1-a2fa637ecbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:24.103404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb1687-0ae5-45d4-97b1-a2fa637ecbac None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:24.103687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb1687-0ae5-45d4-97b1-a2fa637ecbac None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:24.103965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb1687-0ae5-45d4-97b1-a2fa637ecbac None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:24.104703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb1687-0ae5-45d4-97b1-a2fa637ecbac None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:24.106115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb1687-0ae5-45d4-97b1-a2fa637ecbac None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:24.121693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 576/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Tue Jun 21 15:42:23 2022] POST /identity/v3/domains => generated 267 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:42:26.315141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32703b-a820-4f08-8659-5b43b08047c4 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:26.316056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32703b-a820-4f08-8659-5b43b08047c4 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:26.316056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32703b-a820-4f08-8659-5b43b08047c4 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:26.318007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 594/1170] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:26.324585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e2e95-891a-4f9a-b686-13961b5a3f4c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:26.325094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e2e95-891a-4f9a-b686-13961b5a3f4c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:26.325220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e2e95-891a-4f9a-b686-13961b5a3f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:26.344448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-279e2e95-891a-4f9a-b686-13961b5a3f4c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:26.345992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-279e2e95-891a-4f9a-b686-13961b5a3f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:26.384833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:26.419001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 577/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:26.444680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32703b-a820-4f08-8659-5b43b08047c4 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:26.566593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:26.580982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32703b-a820-4f08-8659-5b43b08047c4 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:26.621606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717afaf-b872-4f7f-a930-eefdefa76da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:26.622668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717afaf-b872-4f7f-a930-eefdefa76da6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:26.623001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717afaf-b872-4f7f-a930-eefdefa76da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:26.623284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717afaf-b872-4f7f-a930-eefdefa76da6 None admin] PATH_INFO: `/v3/domains/285f535f94bb4b8bb9cacaf4d21c404c` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:26.635987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717afaf-b872-4f7f-a930-eefdefa76da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=285f535f94bb4b8bb9cacaf4d21c404c)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:26.637803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717afaf-b872-4f7f-a930-eefdefa76da6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:26.640302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 595/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Jun 21 15:42:26 2022] GET /identity/v3/domains/285f535f94bb4b8bb9cacaf4d21c404c => generated 267 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:26.648304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e2e95-891a-4f9a-b686-13961b5a3f4c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:26.753334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e2e95-891a-4f9a-b686-13961b5a3f4c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:26.789111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba80e4-60f6-49f5-8fa7-d32ea05331e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:26.789793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba80e4-60f6-49f5-8fa7-d32ea05331e8 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:26.790138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba80e4-60f6-49f5-8fa7-d32ea05331e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:26.790430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba80e4-60f6-49f5-8fa7-d32ea05331e8 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:26.791171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba80e4-60f6-49f5-8fa7-d32ea05331e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:26.792696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba80e4-60f6-49f5-8fa7-d32ea05331e8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:26.826568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 578/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:42:26 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:42:29.008345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4072d0-8153-4388-9f45-abc42aca8e3e None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:29.008959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4072d0-8153-4388-9f45-abc42aca8e3e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:29.008959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4072d0-8153-4388-9f45-abc42aca8e3e None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:29.010509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 596/1174] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:29.018129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7443eef-0b5e-47df-814f-7edd2d4a858b None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:29.018934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7443eef-0b5e-47df-814f-7edd2d4a858b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:29.019071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7443eef-0b5e-47df-814f-7edd2d4a858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:29.039624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d7443eef-0b5e-47df-814f-7edd2d4a858b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:29.040852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d7443eef-0b5e-47df-814f-7edd2d4a858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:29.059452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:29.116648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 579/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:29.144200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4072d0-8153-4388-9f45-abc42aca8e3e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:29.281132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4072d0-8153-4388-9f45-abc42aca8e3e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:29.323299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68053ce9-3742-4a3f-adef-5a77bfbd031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:29.324286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053ce9-3742-4a3f-adef-5a77bfbd031b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:29.324600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053ce9-3742-4a3f-adef-5a77bfbd031b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:29.324874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053ce9-3742-4a3f-adef-5a77bfbd031b None admin] PATH_INFO: `/v3/domains/285f535f94bb4b8bb9cacaf4d21c404c` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:29.326218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68053ce9-3742-4a3f-adef-5a77bfbd031b None admin] RBAC: Authorizing `identity:get_domain(domain_id=285f535f94bb4b8bb9cacaf4d21c404c)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:29.327828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68053ce9-3742-4a3f-adef-5a77bfbd031b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:29.329812 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 597/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Jun 21 15:42:29 2022] GET /identity/v3/domains/285f535f94bb4b8bb9cacaf4d21c404c => generated 267 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:29.335726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7443eef-0b5e-47df-814f-7edd2d4a858b None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:29.458712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7443eef-0b5e-47df-814f-7edd2d4a858b None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:29.494926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e253287-97b1-46f7-a70c-c062929e044d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:29.495643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253287-97b1-46f7-a70c-c062929e044d None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:29.495918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253287-97b1-46f7-a70c-c062929e044d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:29.496195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e253287-97b1-46f7-a70c-c062929e044d None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:29.496962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e253287-97b1-46f7-a70c-c062929e044d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:29.498463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e253287-97b1-46f7-a70c-c062929e044d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:29.521049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 580/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:42:29 2022] POST /identity/v3/users => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:42:31.660478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dddb0-45f5-485e-ad7b-888153036994 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:31.661003 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dddb0-45f5-485e-ad7b-888153036994 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:31.661003 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dddb0-45f5-485e-ad7b-888153036994 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:31.662360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 598/1178] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:31.668528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10b642-ccb1-4715-b038-af85ab637b12 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:31.669311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10b642-ccb1-4715-b038-af85ab637b12 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:31.669311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10b642-ccb1-4715-b038-af85ab637b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:31.690060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3f10b642-ccb1-4715-b038-af85ab637b12 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:31.691369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3f10b642-ccb1-4715-b038-af85ab637b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:31.695180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:31.781892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 581/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:31.812557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dddb0-45f5-485e-ad7b-888153036994 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:31.838407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:31.925813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dddb0-45f5-485e-ad7b-888153036994 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:31.975048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b299c-813c-490d-9a8e-6f610e999e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:31.975956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b299c-813c-490d-9a8e-6f610e999e8b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:31.976235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b299c-813c-490d-9a8e-6f610e999e8b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:31.976508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b299c-813c-490d-9a8e-6f610e999e8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:31.988931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b299c-813c-490d-9a8e-6f610e999e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-780b299c-813c-490d-9a8e-6f610e999e8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:42:31.997509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:42:31.998626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:42:31.999721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 599/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:42:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:42:32.007471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10b642-ccb1-4715-b038-af85ab637b12 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:32.123973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f10b642-ccb1-4715-b038-af85ab637b12 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:32.163140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:32.163860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:32.164141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:32.164417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:32.165263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:32.165634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:32.167201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:32.177476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 582/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:42:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:32.183155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b299c-813c-490d-9a8e-6f610e999e8b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:32.273133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:32.296108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b299c-813c-490d-9a8e-6f610e999e8b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:32.338412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9df77b-0049-43b9-b11d-9774ab54b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:32.339359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df77b-0049-43b9-b11d-9774ab54b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:32.339700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df77b-0049-43b9-b11d-9774ab54b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:32.340026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df77b-0049-43b9-b11d-9774ab54b8a2 None admin] PATH_INFO: `/v3/users/7cb3b11247b547aba042eee0969e9553` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:32.354543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9df77b-0049-43b9-b11d-9774ab54b8a2 None admin] RBAC: Authorizing `identity:get_user(user_id=7cb3b11247b547aba042eee0969e9553)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:32.356271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9df77b-0049-43b9-b11d-9774ab54b8a2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:32.358333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 600/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:32 2022] GET /identity/v3/users/7cb3b11247b547aba042eee0969e9553 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:32.365717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:32.370739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:32.506166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d395d-a9eb-48d6-86ab-69951d2d4401 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:32.550680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99b301-1cdb-4701-abd4-f4cfa918617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:32.551321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99b301-1cdb-4701-abd4-f4cfa918617d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:32.551604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99b301-1cdb-4701-abd4-f4cfa918617d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:32.551871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99b301-1cdb-4701-abd4-f4cfa918617d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:32.630942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99b301-1cdb-4701-abd4-f4cfa918617d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:32.632412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99b301-1cdb-4701-abd4-f4cfa918617d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:32.634440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:32.743056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 583/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:42:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:42:32.752202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9df77b-0049-43b9-b11d-9774ab54b8a2 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:32.854082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9df77b-0049-43b9-b11d-9774ab54b8a2 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:32.890110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46bbb5-7890-4751-9fca-2d40b19467f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:32.891125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46bbb5-7890-4751-9fca-2d40b19467f5 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:32.891419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46bbb5-7890-4751-9fca-2d40b19467f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:32.891812 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46bbb5-7890-4751-9fca-2d40b19467f5 None admin] PATH_INFO: `/v3/projects/3981e6a682ac483ababf2a6fd8b2a883` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:32.893082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46bbb5-7890-4751-9fca-2d40b19467f5 None admin] RBAC: Authorizing `identity:get_project(project_id=3981e6a682ac483ababf2a6fd8b2a883)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:32.894601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46bbb5-7890-4751-9fca-2d40b19467f5 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:32.896395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 601/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:42:32 2022] GET /identity/v3/projects/3981e6a682ac483ababf2a6fd8b2a883 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:32.903789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99b301-1cdb-4701-abd4-f4cfa918617d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:33.020932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99b301-1cdb-4701-abd4-f4cfa918617d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:33.022519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:33.059034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec9779-78cd-4d48-ba9d-95e7e9262630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:33.059722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec9779-78cd-4d48-ba9d-95e7e9262630 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:33.060010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec9779-78cd-4d48-ba9d-95e7e9262630 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:33.060284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec9779-78cd-4d48-ba9d-95e7e9262630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:33.061152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec9779-78cd-4d48-ba9d-95e7e9262630 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=3981e6a682ac483ababf2a6fd8b2a883, user.id=7cb3b11247b547aba042eee0969e9553) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:33.061464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec9779-78cd-4d48-ba9d-95e7e9262630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:33.063011 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec9779-78cd-4d48-ba9d-95e7e9262630 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:33.068678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 584/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:42:32 2022] GET /identity/v3/role_assignments?user.id=7cb3b11247b547aba042eee0969e9553&scope.project.id=3981e6a682ac483ababf2a6fd8b2a883&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:35.103371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58844ee2-1a0c-472e-9f9f-52429491082f None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:35.103935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58844ee2-1a0c-472e-9f9f-52429491082f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:35.103935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58844ee2-1a0c-472e-9f9f-52429491082f None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:35.105345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 602/1186] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:35.112003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f413b9-f355-4b78-b2cd-80c2618b342e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:35.112483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f413b9-f355-4b78-b2cd-80c2618b342e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:35.112857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f413b9-f355-4b78-b2cd-80c2618b342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:35.133453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-88f413b9-f355-4b78-b2cd-80c2618b342e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:35.134704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-88f413b9-f355-4b78-b2cd-80c2618b342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:35.214924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 585/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:42:35.244140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58844ee2-1a0c-472e-9f9f-52429491082f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:35.253593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:35.354372 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58844ee2-1a0c-472e-9f9f-52429491082f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:35.392769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0483d-47f8-4ec8-b3d0-c1af5ab6aa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:35.393566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0483d-47f8-4ec8-b3d0-c1af5ab6aa04 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:35.393862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0483d-47f8-4ec8-b3d0-c1af5ab6aa04 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:35.394126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0483d-47f8-4ec8-b3d0-c1af5ab6aa04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:35.405192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0483d-47f8-4ec8-b3d0-c1af5ab6aa04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-bea0483d-47f8-4ec8-b3d0-c1af5ab6aa04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:42:35.413075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:42:35.414272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:42:35.415280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 603/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:42:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:42:35.421464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f413b9-f355-4b78-b2cd-80c2618b342e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:35.524411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:35.533143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f413b9-f355-4b78-b2cd-80c2618b342e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:35.576739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:35.577437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:35.577695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:35.578019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:35.578799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:35.579196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:35.580578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:35.588924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 586/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:42:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:42:35.598433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0483d-47f8-4ec8-b3d0-c1af5ab6aa04 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:35.670897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:35.712172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0483d-47f8-4ec8-b3d0-c1af5ab6aa04 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:35.748450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5c592-6388-4ea9-b90c-e802ce36378e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:35.749327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5c592-6388-4ea9-b90c-e802ce36378e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:35.749609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5c592-6388-4ea9-b90c-e802ce36378e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:35.749981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5c592-6388-4ea9-b90c-e802ce36378e None admin] PATH_INFO: `/v3/users/7cb3b11247b547aba042eee0969e9553` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:35.751574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5c592-6388-4ea9-b90c-e802ce36378e None admin] RBAC: Authorizing `identity:get_user(user_id=7cb3b11247b547aba042eee0969e9553)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:35.752995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5c592-6388-4ea9-b90c-e802ce36378e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:35.754678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 604/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:35 2022] GET /identity/v3/users/7cb3b11247b547aba042eee0969e9553 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:35.764674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:35.885879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2b2fe-04f8-4790-be32-32e83a46092b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:35.925662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5db62b-9b0e-4037-82a6-049782662de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:35.926510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5db62b-9b0e-4037-82a6-049782662de5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:35.926793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5db62b-9b0e-4037-82a6-049782662de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:35.927066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5db62b-9b0e-4037-82a6-049782662de5 None admin] PATH_INFO: `/v3/projects/3981e6a682ac483ababf2a6fd8b2a883` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:35.928389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5db62b-9b0e-4037-82a6-049782662de5 None admin] RBAC: Authorizing `identity:get_project(project_id=3981e6a682ac483ababf2a6fd8b2a883)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:35.930033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5db62b-9b0e-4037-82a6-049782662de5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:35.931818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 587/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:42:35 2022] GET /identity/v3/projects/3981e6a682ac483ababf2a6fd8b2a883 => generated 361 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:35.938422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5c592-6388-4ea9-b90c-e802ce36378e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:36.057037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5c592-6388-4ea9-b90c-e802ce36378e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:36.094176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623a3eb-3369-4656-8f70-274ec2c25071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:36.095077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623a3eb-3369-4656-8f70-274ec2c25071 None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:36.095377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623a3eb-3369-4656-8f70-274ec2c25071 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:36.095667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623a3eb-3369-4656-8f70-274ec2c25071 None admin] PATH_INFO: `/v3/projects/3981e6a682ac483ababf2a6fd8b2a883/users/7cb3b11247b547aba042eee0969e9553/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:36.097981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623a3eb-3369-4656-8f70-274ec2c25071 None admin] RBAC: Authorizing `identity:create_grant(project_id=3981e6a682ac483ababf2a6fd8b2a883, user_id=7cb3b11247b547aba042eee0969e9553, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:36.099502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623a3eb-3369-4656-8f70-274ec2c25071 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:36.102146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:36.116325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 605/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:42:35 2022] PUT /identity/v3/projects/3981e6a682ac483ababf2a6fd8b2a883/users/7cb3b11247b547aba042eee0969e9553/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:42:38.160892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c995-9254-49c2-8164-c01148ef01df None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:38.161715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c995-9254-49c2-8164-c01148ef01df None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:38.161849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51c995-9254-49c2-8164-c01148ef01df None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:38.163793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 588/1193] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:42:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:42:38.171686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468e410-f392-4569-a19a-c48290581dce None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:38.172218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468e410-f392-4569-a19a-c48290581dce None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:38.172218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468e410-f392-4569-a19a-c48290581dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:38.195820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-8468e410-f392-4569-a19a-c48290581dce None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:42:38.197050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8468e410-f392-4569-a19a-c48290581dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:42:38.274729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 606/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:42:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:42:38.299138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51c995-9254-49c2-8164-c01148ef01df None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:38.330654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:38.414398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51c995-9254-49c2-8164-c01148ef01df None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:38.457900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76e3cc-7a07-4658-b35d-a97516e45347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:38.461386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76e3cc-7a07-4658-b35d-a97516e45347 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:38.461386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76e3cc-7a07-4658-b35d-a97516e45347 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:38.461386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76e3cc-7a07-4658-b35d-a97516e45347 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:38.473526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76e3cc-7a07-4658-b35d-a97516e45347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-3c76e3cc-7a07-4658-b35d-a97516e45347 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:42:38.481142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:42:38.482300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:42:38.483498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 589/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:42:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:42:38.488666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468e410-f392-4569-a19a-c48290581dce None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:38.595153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468e410-f392-4569-a19a-c48290581dce None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:38.619081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:38.631274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:38.631953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:38.632237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:38.632514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:38.633298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:38.633607 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:38.635062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:38.643776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 607/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:42:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:38.650256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76e3cc-7a07-4658-b35d-a97516e45347 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:38.746128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:38.763607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76e3cc-7a07-4658-b35d-a97516e45347 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:38.821549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df82e2-abbb-4172-8f9c-6a9879d3dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:38.822448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df82e2-abbb-4172-8f9c-6a9879d3dcce None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:38.822724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df82e2-abbb-4172-8f9c-6a9879d3dcce None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:38.823000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df82e2-abbb-4172-8f9c-6a9879d3dcce None admin] PATH_INFO: `/v3/users/7cb3b11247b547aba042eee0969e9553` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:38.824302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df82e2-abbb-4172-8f9c-6a9879d3dcce None admin] RBAC: Authorizing `identity:get_user(user_id=7cb3b11247b547aba042eee0969e9553)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:38.825739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df82e2-abbb-4172-8f9c-6a9879d3dcce None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:38.827425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 590/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Jun 21 15:42:38 2022] GET /identity/v3/users/7cb3b11247b547aba042eee0969e9553 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:38.837102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:38.944790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593a2f5-60c1-4f2e-aae2-9b4bfdb18f2c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:38.982957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503a125-ce35-4e46-a0bb-539e3129e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:38.983631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a125-ce35-4e46-a0bb-539e3129e76d None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:38.983921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a125-ce35-4e46-a0bb-539e3129e76d None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:38.984370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a125-ce35-4e46-a0bb-539e3129e76d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:39.040560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:39.055732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503a125-ce35-4e46-a0bb-539e3129e76d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:39.057257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503a125-ce35-4e46-a0bb-539e3129e76d None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:39.175265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 608/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:42:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:42:39.181628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df82e2-abbb-4172-8f9c-6a9879d3dcce None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:39.302832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df82e2-abbb-4172-8f9c-6a9879d3dcce None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:39.341981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5962f3d5-0e3a-4807-a7f5-50091ba75fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:39.342973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962f3d5-0e3a-4807-a7f5-50091ba75fe1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:39.343326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962f3d5-0e3a-4807-a7f5-50091ba75fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:39.343691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5962f3d5-0e3a-4807-a7f5-50091ba75fe1 None admin] PATH_INFO: `/v3/projects/3981e6a682ac483ababf2a6fd8b2a883` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:39.344973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962f3d5-0e3a-4807-a7f5-50091ba75fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=3981e6a682ac483ababf2a6fd8b2a883)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:39.346548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5962f3d5-0e3a-4807-a7f5-50091ba75fe1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:39.348447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 591/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Jun 21 15:42:39 2022] GET /identity/v3/projects/3981e6a682ac483ababf2a6fd8b2a883 => generated 361 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:39.359246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503a125-ce35-4e46-a0bb-539e3129e76d None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:42:39.464122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:39.482729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503a125-ce35-4e46-a0bb-539e3129e76d None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:42:39.519819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b67c6f-372f-4c3d-b8e2-5664b803bb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:42:39.520478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67c6f-372f-4c3d-b8e2-5664b803bb64 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:42:39.520803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67c6f-372f-4c3d-b8e2-5664b803bb64 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:42:39.521074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b67c6f-372f-4c3d-b8e2-5664b803bb64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:42:39.521946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b67c6f-372f-4c3d-b8e2-5664b803bb64 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=3981e6a682ac483ababf2a6fd8b2a883, user.id=7cb3b11247b547aba042eee0969e9553) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:42:39.522263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b67c6f-372f-4c3d-b8e2-5664b803bb64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:42:39.523746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b67c6f-372f-4c3d-b8e2-5664b803bb64 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:42:39.530071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 609/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:42:39 2022] GET /identity/v3/role_assignments?user.id=7cb3b11247b547aba042eee0969e9553&scope.project.id=3981e6a682ac483ababf2a6fd8b2a883&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:42:49.336377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:42:49.527348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=25 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:36.244008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798508a-9c8e-413a-b041-a89fc6be4085 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:36.244008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798508a-9c8e-413a-b041-a89fc6be4085 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:36.244008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798508a-9c8e-413a-b041-a89fc6be4085 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:36.245951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 592/1201] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:43:36.252462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec92f-e94d-4231-bd95-1ddf04748b69 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:36.253515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec92f-e94d-4231-bd95-1ddf04748b69 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:36.253950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec92f-e94d-4231-bd95-1ddf04748b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:36.274479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d76ec92f-e94d-4231-bd95-1ddf04748b69 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:43:36.276243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d76ec92f-e94d-4231-bd95-1ddf04748b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:43:36.377818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 610/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:43:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:43:36.407566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798508a-9c8e-413a-b041-a89fc6be4085 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:36.547147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798508a-9c8e-413a-b041-a89fc6be4085 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:36.609947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:36.611266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:36.611623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:36.612035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:36.626462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:36.627939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:36.649926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:36.651329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:36.659044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:36.659044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:36.659044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:36.659044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:36.659044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 593/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:36.661089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec92f-e94d-4231-bd95-1ddf04748b69 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:36.803653 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec92f-e94d-4231-bd95-1ddf04748b69 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:36.839717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:36.850246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42cd6d0-f778-46f1-a08c-b0f6034c34c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:36.850950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cd6d0-f778-46f1-a08c-b0f6034c34c1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:36.851272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cd6d0-f778-46f1-a08c-b0f6034c34c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:36.851569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42cd6d0-f778-46f1-a08c-b0f6034c34c1 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:36.852542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42cd6d0-f778-46f1-a08c-b0f6034c34c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:36.852788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42cd6d0-f778-46f1-a08c-b0f6034c34c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:36.854433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42cd6d0-f778-46f1-a08c-b0f6034c34c1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:36.874200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 611/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:36.875954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:36.971955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:37.048489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b4bcc-7d7f-44ab-a6c3-7e4901b7c8c5 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:37.122973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacdf39-f50a-420a-acf1-b586a1e31297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:37.123839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacdf39-f50a-420a-acf1-b586a1e31297 None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:37.124240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacdf39-f50a-420a-acf1-b586a1e31297 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:37.124703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacdf39-f50a-420a-acf1-b586a1e31297 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:37.125858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacdf39-f50a-420a-acf1-b586a1e31297 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:37.128623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacdf39-f50a-420a-acf1-b586a1e31297 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:37.168195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 594/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Jun 21 15:43:36 2022] POST /identity/v3/users => generated 251 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:43:39.430543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64ce02-9ae2-4a24-9285-d40ae4f15f9b None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:39.431474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64ce02-9ae2-4a24-9285-d40ae4f15f9b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:39.431474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64ce02-9ae2-4a24-9285-d40ae4f15f9b None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:39.432717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 612/1206] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:43:39.440566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238c9d-b3cb-4e28-87e7-f3bf358f8488 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:39.441634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238c9d-b3cb-4e28-87e7-f3bf358f8488 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:39.441634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65238c9d-b3cb-4e28-87e7-f3bf358f8488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:39.467637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-65238c9d-b3cb-4e28-87e7-f3bf358f8488 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:43:39.468912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-65238c9d-b3cb-4e28-87e7-f3bf358f8488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:43:39.569195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 595/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:43:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:43:39.603992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64ce02-9ae2-4a24-9285-d40ae4f15f9b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:39.754689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64ce02-9ae2-4a24-9285-d40ae4f15f9b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:39.805685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93981fa-a842-45fb-a6e5-348873a0b27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:39.806519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93981fa-a842-45fb-a6e5-348873a0b27c None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:39.806797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93981fa-a842-45fb-a6e5-348873a0b27c None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:39.807064 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93981fa-a842-45fb-a6e5-348873a0b27c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:39.821074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93981fa-a842-45fb-a6e5-348873a0b27c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-e93981fa-a842-45fb-a6e5-348873a0b27c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:43:39.831611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:39.832626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:39.833483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 613/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:43:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:39.840909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238c9d-b3cb-4e28-87e7-f3bf358f8488 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:39.891997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:39.988822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65238c9d-b3cb-4e28-87e7-f3bf358f8488 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:40.035237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:40.035929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:40.036216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:40.036476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:40.037227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:40.037569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:40.039232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:40.060273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 596/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:43:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:40.063329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93981fa-a842-45fb-a6e5-348873a0b27c None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:40.138300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:40.204914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93981fa-a842-45fb-a6e5-348873a0b27c None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:40.256390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:40.257475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:40.258151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:40.258151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:40.272455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:40.273964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:40.295935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:40.297077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:40.298289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:40.298289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:40.298289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:40.298289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:40.298289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 614/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:40.306281 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:40.437462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:40.442104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c6835-ddcf-4698-829b-0adb232defd8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:40.497080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:40.498352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:40.498880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:40.499369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:40.500527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:40.500788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:40.502273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:40.517257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 597/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:40.523403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:40.672471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9dc22-3ad6-41ff-8525-2fbdab588306 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:40.674911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:40.723176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:40.724545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:40.724860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:40.725137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:40.750027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:40.751902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:43:40.778313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:40.779619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:40.790476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 615/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:40.791455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:40.927750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178758d-658a-4369-9d5f-bb56f68c133d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:40.979866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:40.980751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:40.981043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:40.981337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:40.982383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:40.982803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:40.984456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:41.005214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 598/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jun 21 15:43:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:41.011954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:41.159828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a582-9e6b-4ece-bed6-0bb76bf963bc None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:41.208511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1112f96-527e-4fd8-b764-c00e57da2097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:41.209180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1112f96-527e-4fd8-b764-c00e57da2097 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:41.209464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1112f96-527e-4fd8-b764-c00e57da2097 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:41.209742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1112f96-527e-4fd8-b764-c00e57da2097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:41.250301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:41.321803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1112f96-527e-4fd8-b764-c00e57da2097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:41.323706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1112f96-527e-4fd8-b764-c00e57da2097 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:41.482389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 616/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:43:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:43:41.488802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:41.511969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:41.621998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95138834-9524-49d0-b6e0-dbdb20911251 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:41.666545 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:41.667758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:41.668112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:41.668488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:41.681591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:41.683511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:41.705366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:41.706544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:41.707623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:41.707623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:41.707623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:41.707623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:41.708424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 599/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:43:41.715545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1112f96-527e-4fd8-b764-c00e57da2097 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:41.788408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:41.837157 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1112f96-527e-4fd8-b764-c00e57da2097 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:41.879806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:41.880543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:41.880852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:41.881137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:41.881894 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:41.882496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:41.884002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:41.900694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 617/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:41.907387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:42.003062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:42.055102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594763d-12a1-4657-8641-a2f5e49de502 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:42.110137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:42.111107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:42.111489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:42.111865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:42.119018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:42.120682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:43:42.138090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:42.139279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:42.140342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:43:42.140342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:43:42.140342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:43:42.140342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:43:42.140342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:43:42.140342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:43:42.140342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:43:42.140342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:42.147320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 600/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:43:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:42.149069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:42.266866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0543b9-fcd6-4bbe-99f0-946eb74f1aa6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:42.293738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:42.308590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073565a-a1fe-4a2d-a6fd-86ce6cccd3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:42.309545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073565a-a1fe-4a2d-a6fd-86ce6cccd3dd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:42.310109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073565a-a1fe-4a2d-a6fd-86ce6cccd3dd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:42.310510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073565a-a1fe-4a2d-a6fd-86ce6cccd3dd None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:42.311389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073565a-a1fe-4a2d-a6fd-86ce6cccd3dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:42.311796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073565a-a1fe-4a2d-a6fd-86ce6cccd3dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:42.313342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073565a-a1fe-4a2d-a6fd-86ce6cccd3dd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:42.327215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 618/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:43:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:42.333134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:42.456180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ecc9a-5d25-4d60-a175-50b12c472c8f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:42.496926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:42.503434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96bc18-9f94-4390-8e81-e1815047df49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:42.504097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96bc18-9f94-4390-8e81-e1815047df49 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:42.504373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96bc18-9f94-4390-8e81-e1815047df49 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:42.504641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96bc18-9f94-4390-8e81-e1815047df49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:42.505471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96bc18-9f94-4390-8e81-e1815047df49 None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=16defb911ffa485da9164fa2674baba1) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:42.505798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96bc18-9f94-4390-8e81-e1815047df49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:42.507889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96bc18-9f94-4390-8e81-e1815047df49 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:42.525807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 601/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:43:42 2022] GET /identity/v3/role_assignments?user.id=16defb911ffa485da9164fa2674baba1&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:44.581557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d646d-b621-4dbf-bdc2-f603fea70b61 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:44.582142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d646d-b621-4dbf-bdc2-f603fea70b61 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:44.582142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d646d-b621-4dbf-bdc2-f603fea70b61 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:44.583696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 619/1220] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:43:44.590130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f6015-3915-4874-846d-dd5c046610d3 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:44.590756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f6015-3915-4874-846d-dd5c046610d3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:44.590860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f6015-3915-4874-846d-dd5c046610d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:44.612727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ba7f6015-3915-4874-846d-dd5c046610d3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:43:44.612727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ba7f6015-3915-4874-846d-dd5c046610d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:43:44.690009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 602/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:43:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:43:44.713441 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d646d-b621-4dbf-bdc2-f603fea70b61 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:44.834305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d646d-b621-4dbf-bdc2-f603fea70b61 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:44.882037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f950e-34fa-4ee2-bf93-22926e7bacc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:44.882934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f950e-34fa-4ee2-bf93-22926e7bacc1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:44.883221 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f950e-34fa-4ee2-bf93-22926e7bacc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:44.883494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f950e-34fa-4ee2-bf93-22926e7bacc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:44.898004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f950e-34fa-4ee2-bf93-22926e7bacc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-997f950e-34fa-4ee2-bf93-22926e7bacc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:43:44.907186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:44.909901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:44.910831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 620/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:43:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:44.915673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f6015-3915-4874-846d-dd5c046610d3 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:45.048351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f6015-3915-4874-846d-dd5c046610d3 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:45.074600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:45.086895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:45.087490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:45.087829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:45.088055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:45.088797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:45.089126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:45.090547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:45.100159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 603/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:43:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:45.107294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f950e-34fa-4ee2-bf93-22926e7bacc1 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:45.163824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:45.227051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f950e-34fa-4ee2-bf93-22926e7bacc1 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:45.269943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:45.270786 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:45.271149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:45.271419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:45.283334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:45.284933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:45.306197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:45.307470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:45.308510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:45.308510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:45.308510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:45.308510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:45.308510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 621/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:45.314656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:45.426191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05a50e-c7f1-415a-bf87-f9e6019110d2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:45.463289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:45.463964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:45.464246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:45.464543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:45.465302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:45.465688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:45.467089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:45.480051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 604/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:45.487222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:45.602849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:45.610648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a4c99-78a1-41e8-aeda-c79467622444 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:45.650486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:45.651266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:45.651553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:45.651862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:45.660762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:45.662272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:43:45.676754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:45.677742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:45.678736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 622/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:45.683921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:45.738481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:45.808927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bcd34-4a22-4906-bb02-af146a2325fa None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:45.848518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:45.849309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:45.849637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:45.850011 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:45.850965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:45.851365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:45.853282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:45.879471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 605/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jun 21 15:43:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:45.886818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:46.036557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760d7e9-8811-4dbe-927d-56936c4a1a06 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:46.077964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:46.078829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:46.079110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:46.079381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:46.092572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:46.094365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:46.114616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:46.124806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:46.126232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:46.126232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:46.126232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:46.126232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:46.126232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 623/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:43:46.126622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:46.242962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64184a64-45dd-4284-857d-d9f36511bc0c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:46.246092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:46.285055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:46.285727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:46.286048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:46.286328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:46.287160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:46.287412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:46.288890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:46.302023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 606/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:46.307795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:46.318145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:46.429550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f9454-624d-4dd0-af71-fa420080fbf4 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:46.471957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:46.473093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:46.473419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:46.473713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:46.482137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:46.483672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:43:46.500454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:46.501604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:46.502608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 624/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:46.508319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:46.629260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99a4e5-4312-458f-84f8-8972c10f7da1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:46.675028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5156ce8-043f-49b1-8bed-1e338a223e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:46.677366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156ce8-043f-49b1-8bed-1e338a223e01 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:46.677366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156ce8-043f-49b1-8bed-1e338a223e01 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:46.677366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156ce8-043f-49b1-8bed-1e338a223e01 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:46.677366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5156ce8-043f-49b1-8bed-1e338a223e01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:46.677734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5156ce8-043f-49b1-8bed-1e338a223e01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:46.679203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5156ce8-043f-49b1-8bed-1e338a223e01 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:46.697991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 607/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:43:46.701112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:46.782698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:46.823631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64abbf3b-8300-45eb-af07-734af7a93171 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:46.867623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5b220-06a8-416a-aca5-0c6a190f543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:46.868537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5b220-06a8-416a-aca5-0c6a190f543f None admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:46.868822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5b220-06a8-416a-aca5-0c6a190f543f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:46.869124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5b220-06a8-416a-aca5-0c6a190f543f None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/16defb911ffa485da9164fa2674baba1/roles/7a558a55cb054e62ac77df614c5bd73d` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:46.886934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5b220-06a8-416a-aca5-0c6a190f543f None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=16defb911ffa485da9164fa2674baba1, role_id=7a558a55cb054e62ac77df614c5bd73d)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:46.888671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5b220-06a8-416a-aca5-0c6a190f543f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:46.902709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 625/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:43:46 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/16defb911ffa485da9164fa2674baba1/roles/7a558a55cb054e62ac77df614c5bd73d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:43:49.267013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097ce3-352c-4b64-b4fd-8ab094ba1152 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:49.267662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097ce3-352c-4b64-b4fd-8ab094ba1152 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:49.267662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db097ce3-352c-4b64-b4fd-8ab094ba1152 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:49.269000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 608/1233] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:43:49.274697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3565236-d0e4-4e0e-a9d7-2e446eb9c574 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:49.275465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3565236-d0e4-4e0e-a9d7-2e446eb9c574 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:49.275465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3565236-d0e4-4e0e-a9d7-2e446eb9c574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:49.296681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d3565236-d0e4-4e0e-a9d7-2e446eb9c574 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:43:49.298333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d3565236-d0e4-4e0e-a9d7-2e446eb9c574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:43:49.380649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 626/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:43:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:43:49.406202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db097ce3-352c-4b64-b4fd-8ab094ba1152 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:49.434720 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:49.537799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db097ce3-352c-4b64-b4fd-8ab094ba1152 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:49.578048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a22d-52ff-4584-8fcb-a83ac180b81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:49.578945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a22d-52ff-4584-8fcb-a83ac180b81a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:49.579226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a22d-52ff-4584-8fcb-a83ac180b81a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:49.579598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f731a22d-52ff-4584-8fcb-a83ac180b81a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:49.591649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f731a22d-52ff-4584-8fcb-a83ac180b81a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-f731a22d-52ff-4584-8fcb-a83ac180b81a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:43:49.599625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:49.601098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:49.602081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 609/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Jun 21 15:43:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:43:49.607697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3565236-d0e4-4e0e-a9d7-2e446eb9c574 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:49.661549 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:49.724105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3565236-d0e4-4e0e-a9d7-2e446eb9c574 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:49.764871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:49.765638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:49.765953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:49.766232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:49.766995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] RBAC: Adding query filter params (name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:49.767349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:49.768983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:49.780525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 627/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Jun 21 15:43:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:49.786527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a22d-52ff-4584-8fcb-a83ac180b81a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:49.895362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:49.914427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f731a22d-52ff-4584-8fcb-a83ac180b81a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:49.957440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:49.958366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:49.958690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:49.959008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:49.972968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:49.974663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:50.011366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:50.012566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:50.013644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:50.013644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:50.013644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:50.013644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:50.013908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 610/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:50.020998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:50.141100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016a919-c60d-4843-badf-f8ab751fac94 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:50.153125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:50.185355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:50.186076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:50.186358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:50.186628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:50.187358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:50.187661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:50.189423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:50.206425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 628/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:50.213134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:50.343116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcef71-cda0-4952-a114-83770c912211 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:50.389141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:50.390094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:50.390373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:50.390829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] PATH_INFO: `/v3/users/placement` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:50.394348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:50.403834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:50.405542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:43:50.422135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:50.424309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:50.425530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 611/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:50.430029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:50.554185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438842d6-3fea-4a28-b7f2-91f89c043bb9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:50.594155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:50.594850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:50.595127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:50.595583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:50.596621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:50.597100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:50.598870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:50.625504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 629/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jun 21 15:43:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:50.633147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:50.758781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d303900-dfa1-40dd-a0a8-35feb19ef53d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:50.800231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd43c17-9d9d-42ee-b757-f5b9bb215fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:50.800903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd43c17-9d9d-42ee-b757-f5b9bb215fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:50.801243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd43c17-9d9d-42ee-b757-f5b9bb215fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:50.801554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd43c17-9d9d-42ee-b757-f5b9bb215fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:50.885816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:50.897693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd43c17-9d9d-42ee-b757-f5b9bb215fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:50.899572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd43c17-9d9d-42ee-b757-f5b9bb215fb3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:51.036835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 612/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:43:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 21 15:43:51.043294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:51.073061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:51.160862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3a448-01ed-4e6b-b15c-c0ccdb6d7487 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:51.198713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:51.199520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:51.199799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:51.200070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:51.211964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:51.213828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:51.233449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:51.234555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:51.235482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:51.235482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:51.235482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:51.235482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:51.235482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 630/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:51.241453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd43c17-9d9d-42ee-b757-f5b9bb215fb3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:51.363767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:51.370125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd43c17-9d9d-42ee-b757-f5b9bb215fb3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:51.413978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:51.414670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:51.414941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:51.415405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:51.416276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:51.416582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:51.417996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:51.431819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 613/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:51.438629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:51.532007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:51.562228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe1698-cf11-4740-918e-ff307a621917 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:51.601264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:51.602233 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:51.602513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:51.602710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:51.611184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:51.612787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:43:51.628248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:51.629473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:51.630602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 631/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:43:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:51.636892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:51.759802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3703a0-3ed2-4be7-945e-abccd552eba1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:51.798142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5929a-d503-483f-9d5b-a51ec95e8666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:51.798813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5929a-d503-483f-9d5b-a51ec95e8666 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:51.799094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5929a-d503-483f-9d5b-a51ec95e8666 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:51.799368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5929a-d503-483f-9d5b-a51ec95e8666 None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:51.800161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5929a-d503-483f-9d5b-a51ec95e8666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:51.800467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5929a-d503-483f-9d5b-a51ec95e8666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:51.801939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5929a-d503-483f-9d5b-a51ec95e8666 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:51.820000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 614/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:43:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:51.827334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:51.963266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a5051-c636-4019-b81f-3dfb7bc3965a None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:52.011688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5d7a2-7076-432f-8059-fb443bfa121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:52.012513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5d7a2-7076-432f-8059-fb443bfa121a None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:52.012723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5d7a2-7076-432f-8059-fb443bfa121a None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:52.012999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5d7a2-7076-432f-8059-fb443bfa121a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:52.014059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5d7a2-7076-432f-8059-fb443bfa121a None admin] RBAC: Adding query filter params (role.id=7a558a55cb054e62ac77df614c5bd73d, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=16defb911ffa485da9164fa2674baba1) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:52.014544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5d7a2-7076-432f-8059-fb443bfa121a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:52.016434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5d7a2-7076-432f-8059-fb443bfa121a None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:52.023270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 632/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:43:51 2022] GET /identity/v3/role_assignments?user.id=16defb911ffa485da9164fa2674baba1&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=7a558a55cb054e62ac77df614c5bd73d => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:54.456931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4d0b-f367-43c7-bcdd-d4a988e56468 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:54.457575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4d0b-f367-43c7-bcdd-d4a988e56468 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:54.457694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4d0b-f367-43c7-bcdd-d4a988e56468 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:54.459146 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 615/1247] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:43:54.465268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618d4d3-5234-408f-ba7e-10ecc6414651 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:54.465959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618d4d3-5234-408f-ba7e-10ecc6414651 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:54.465959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618d4d3-5234-408f-ba7e-10ecc6414651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:54.471743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:54.485455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5618d4d3-5234-408f-ba7e-10ecc6414651 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:43:54.486921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5618d4d3-5234-408f-ba7e-10ecc6414651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:43:54.576455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 633/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:43:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:43:54.601306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae4d0b-f367-43c7-bcdd-d4a988e56468 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:54.624409 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:54.726311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae4d0b-f367-43c7-bcdd-d4a988e56468 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:54.769027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390e0be-4a4a-4223-95f7-903de3fad47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:54.769893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390e0be-4a4a-4223-95f7-903de3fad47f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:54.770161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390e0be-4a4a-4223-95f7-903de3fad47f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:54.770435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390e0be-4a4a-4223-95f7-903de3fad47f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:54.782051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390e0be-4a4a-4223-95f7-903de3fad47f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-5390e0be-4a4a-4223-95f7-903de3fad47f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:43:54.791740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:54.793251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:54.794347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 616/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:43:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:54.799383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618d4d3-5234-408f-ba7e-10ecc6414651 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:54.920384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5618d4d3-5234-408f-ba7e-10ecc6414651 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:54.956605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:54.964829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:54.965689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:54.966008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:54.966803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:54.967740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:54.967928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:54.969647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:54.980641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 634/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:43:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:54.986661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390e0be-4a4a-4223-95f7-903de3fad47f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:55.080171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:55.106717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390e0be-4a4a-4223-95f7-903de3fad47f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:55.146329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:55.147137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:55.147408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:55.147677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:55.160374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:55.161886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:55.182363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:55.189045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:55.190761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:55.190761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:55.190761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:55.190761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:55.190761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 617/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:55.192382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:55.318513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eecc72-0874-4543-a50f-d22c047a3fb2 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:55.357726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:55.358697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:55.359121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:55.359555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:55.360292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:55.360636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:55.362024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:55.384420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 635/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:43:55.390698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:55.512793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9e398-e4bd-4605-a983-09068419467b None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:55.542593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:55.551171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:55.552090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:55.552409 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:55.552753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] PATH_INFO: `/v3/users/placement` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:55.563319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:55.565464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:43:55.584249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:55.585506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:55.586663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 618/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:55.592758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:55.656428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:55.707443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef948e1e-0eb1-4d50-91a4-1a96ea7a8f5b None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:55.748450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:55.749197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:55.749508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:55.749832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:55.750727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:55.751040 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:55.752582 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:55.774937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 636/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jun 21 15:43:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:55.781077 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:55.918520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd33e33-eb40-4f46-bceb-b46b2ba1324d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:55.962076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829c33f-3162-4efa-999b-4fc61b7f9f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:55.962969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829c33f-3162-4efa-999b-4fc61b7f9f6f None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:55.963272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829c33f-3162-4efa-999b-4fc61b7f9f6f None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:55.963527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829c33f-3162-4efa-999b-4fc61b7f9f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:56.050561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:56.066649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829c33f-3162-4efa-999b-4fc61b7f9f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:56.068134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829c33f-3162-4efa-999b-4fc61b7f9f6f None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:56.192057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 619/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:43:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 15:43:56.199040 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:56.336060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e743b-fb8e-4b3f-9d48-59cb39f92f76 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:56.356762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:56.382441 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:56.383377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:56.383695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:56.384072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:56.396205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:56.397906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:56.422369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:56.423569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:56.429706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:56.429706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:56.429706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:56.429706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:56.429706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 637/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:43:56.432390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829c33f-3162-4efa-999b-4fc61b7f9f6f None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:56.539909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:56.553398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829c33f-3162-4efa-999b-4fc61b7f9f6f None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:56.597883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:56.598638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:56.599048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:56.599415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:56.600589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:56.601060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:56.603081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:56.618205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 620/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:56.625942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:56.763133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9a092-4ad6-4498-8629-c08b07f52ec6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:56.804087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:56.804898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:56.805186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:56.805478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:56.813148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:56.814749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:43:56.833139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:56.835176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:56.837371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:43:56.837371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:43:56.837371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:43:56.837371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:43:56.837371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:43:56.837371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:43:56.837371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:43:56.837371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:43:56.838292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 638/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:43:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:56.844128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:56.978859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecf350-56dd-41dd-aeaf-44b277c428b1 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:57.020547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3df4e-bd33-43c6-b199-c81cfacc4fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:57.021295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3df4e-bd33-43c6-b199-c81cfacc4fbc None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:57.021663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3df4e-bd33-43c6-b199-c81cfacc4fbc None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:57.022052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3df4e-bd33-43c6-b199-c81cfacc4fbc None admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:57.022914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3df4e-bd33-43c6-b199-c81cfacc4fbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:57.023278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3df4e-bd33-43c6-b199-c81cfacc4fbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:57.025226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3df4e-bd33-43c6-b199-c81cfacc4fbc None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:57.038175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 621/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:43:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:43:57.045709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:57.077257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:57.162896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cba1a-ef7c-4bd6-81ca-09bd59b20f1e None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:57.201410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfd79a-9ff4-4ce7-8e29-a314bca909dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:57.202145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfd79a-9ff4-4ce7-8e29-a314bca909dd None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:57.202432 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfd79a-9ff4-4ce7-8e29-a314bca909dd None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:57.202729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfd79a-9ff4-4ce7-8e29-a314bca909dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:57.203584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfd79a-9ff4-4ce7-8e29-a314bca909dd None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=16defb911ffa485da9164fa2674baba1) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:57.203900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfd79a-9ff4-4ce7-8e29-a314bca909dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:57.205379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfd79a-9ff4-4ce7-8e29-a314bca909dd None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:57.211778 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 639/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:43:57 2022] GET /identity/v3/role_assignments?user.id=16defb911ffa485da9164fa2674baba1&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:59.295394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0ce4d-ae48-4e2c-ade5-8c1a80f42fc5 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:59.295972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0ce4d-ae48-4e2c-ade5-8c1a80f42fc5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:59.295972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0ce4d-ae48-4e2c-ade5-8c1a80f42fc5 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:59.297559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 622/1261] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:43:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:43:59.303551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864d30-bd1b-4878-bb55-b4620ddef1ab None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:59.303994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864d30-bd1b-4878-bb55-b4620ddef1ab None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:59.304516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864d30-bd1b-4878-bb55-b4620ddef1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:59.327170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-07864d30-bd1b-4878-bb55-b4620ddef1ab None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:43:59.328461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-07864d30-bd1b-4878-bb55-b4620ddef1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:43:59.406546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 640/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:43:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:43:59.429813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0ce4d-ae48-4e2c-ade5-8c1a80f42fc5 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:59.441755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:59.552249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0ce4d-ae48-4e2c-ade5-8c1a80f42fc5 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:59.588140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc828-ced4-43ad-ac02-c4a588f682b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:59.588925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc828-ced4-43ad-ac02-c4a588f682b2 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:59.589204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc828-ced4-43ad-ac02-c4a588f682b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:59.589477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afc828-ced4-43ad-ac02-c4a588f682b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:59.599881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afc828-ced4-43ad-ac02-c4a588f682b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-89afc828-ced4-43ad-ac02-c4a588f682b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:43:59.607572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:59.608959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:59.609843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 623/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:43:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:59.615295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864d30-bd1b-4878-bb55-b4620ddef1ab None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:59.634463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:59.720928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864d30-bd1b-4878-bb55-b4620ddef1ab None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:59.760613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:59.761450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:59.761885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:59.762363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:59.763109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:43:59.763416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:59.764802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:59.773289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 641/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:43:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:43:59.779847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc828-ced4-43ad-ac02-c4a588f682b2 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:43:59.854321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:43:59.894635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afc828-ced4-43ad-ac02-c4a588f682b2 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:43:59.931779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:43:59.933557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:43:59.934151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:43:59.934577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:43:59.950190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:43:59.952306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:43:59.983324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:43:59.984787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:43:59.986525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:43:59.986525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:43:59.986525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:43:59.986525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:43:59.986525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 624/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:43:59.992131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:00.087691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:00.109201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3996c0-2578-4f83-b9b9-60887af05288 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:00.145042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:00.145708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:00.146018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:00.146297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:00.147022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:00.147332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:00.148727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:00.162269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 642/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:00.169340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:00.298863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30a616-ef02-4aec-8ce7-7922a1ee26a8 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:00.318685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:00.335911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:00.336866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:00.337187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:00.337523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:00.345644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:00.347130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:44:00.361728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:00.362824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:44:00.363801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 625/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:44:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:00.369005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:00.477475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7f334-3839-4295-96a9-e08f8a53a567 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:00.513499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:00.514193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:00.514473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:00.514769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:00.515645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:00.515955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:00.517406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:00.526514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:00.536039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 643/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jun 21 15:44:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:00.542262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:00.650467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d1d21-9b2a-43bf-842d-4d7b68de706a None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:00.685444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:00.686260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:00.686539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:00.686808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:00.698025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:00.699485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:44:00.716764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:00.718437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:00.720092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:44:00.720092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:44:00.720092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:44:00.720092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:44:00.720092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 626/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:00.724810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:00.848921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d5e27-5b2a-446f-bfcd-83faa753f05f None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:00.885400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:00.886094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:00.886377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:00.886650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:00.887368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:00.887674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:00.889048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:00.900754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 644/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:00.906705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:00.945206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:01.031981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ccd6d-079a-4b80-a69a-8a02565d7472 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:01.069581 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:01.070402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:01.070679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:01.071028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:01.078100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:01.079575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:44:01.094268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:01.095297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:44:01.096569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 627/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:44:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:01.101390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:01.165836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:01.209060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85727833-0e55-4339-ac26-2ce9239ccc29 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:01.243949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e7929-ceaf-45a0-bf2a-dee1fabde781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:01.244631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7929-ceaf-45a0-bf2a-dee1fabde781 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:01.244917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7929-ceaf-45a0-bf2a-dee1fabde781 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:01.245193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7929-ceaf-45a0-bf2a-dee1fabde781 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:01.246023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e7929-ceaf-45a0-bf2a-dee1fabde781 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:01.246333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e7929-ceaf-45a0-bf2a-dee1fabde781 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:01.247791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e7929-ceaf-45a0-bf2a-dee1fabde781 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:01.261286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 645/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:44:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:01.267608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:01.370151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:01.382615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2532c-6b30-4429-9ef8-273e4516aba3 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:01.419521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc733147-f2e5-41d6-b428-a15224a7ef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:01.420333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc733147-f2e5-41d6-b428-a15224a7ef73 None admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:01.420614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc733147-f2e5-41d6-b428-a15224a7ef73 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:01.420892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc733147-f2e5-41d6-b428-a15224a7ef73 None admin] PATH_INFO: `/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/16defb911ffa485da9164fa2674baba1/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:01.422977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc733147-f2e5-41d6-b428-a15224a7ef73 None admin] RBAC: Authorizing `identity:create_grant(project_id=60408c00fa3d4698b251ca2a59ae8c5f, user_id=16defb911ffa485da9164fa2674baba1, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:01.424427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc733147-f2e5-41d6-b428-a15224a7ef73 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:01.440362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 628/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Jun 21 15:44:01 2022] PUT /identity/v3/projects/60408c00fa3d4698b251ca2a59ae8c5f/users/16defb911ffa485da9164fa2674baba1/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:44:03.489619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23f142-9b7c-4837-8087-9389d7863393 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:03.490199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23f142-9b7c-4837-8087-9389d7863393 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:03.490199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23f142-9b7c-4837-8087-9389d7863393 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:03.491974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 646/1274] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:44:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:44:03.499799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634be60-9da6-4395-ba1f-c8bc60be61fe None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:03.500281 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634be60-9da6-4395-ba1f-c8bc60be61fe None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:03.500281 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634be60-9da6-4395-ba1f-c8bc60be61fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:03.518347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1634be60-9da6-4395-ba1f-c8bc60be61fe None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:44:03.519577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1634be60-9da6-4395-ba1f-c8bc60be61fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:44:03.594875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 629/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:44:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 21 15:44:03.619365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23f142-9b7c-4837-8087-9389d7863393 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:03.728132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23f142-9b7c-4837-8087-9389d7863393 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:03.746231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:03.768111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ca041-5362-4270-b8ae-67bc92e68437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:03.768846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ca041-5362-4270-b8ae-67bc92e68437 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:03.769124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ca041-5362-4270-b8ae-67bc92e68437 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:03.769395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ca041-5362-4270-b8ae-67bc92e68437 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:03.780218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ca041-5362-4270-b8ae-67bc92e68437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-6b0ca041-5362-4270-b8ae-67bc92e68437 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 21 15:44:03.788183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:03.789258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 21 15:44:03.790438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:03.793962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 647/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Jun 21 15:44:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:03.795637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634be60-9da6-4395-ba1f-c8bc60be61fe None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:03.885043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:03.908753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634be60-9da6-4395-ba1f-c8bc60be61fe None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:03.953563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:03.954565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:03.955033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:03.955488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:03.956746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:03.957249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:03.959376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:03.973368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 630/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Jun 21 15:44:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:03.980126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ca041-5362-4270-b8ae-67bc92e68437 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:04.099390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ca041-5362-4270-b8ae-67bc92e68437 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:04.136994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:04.137811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:04.138115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:04.138402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:04.150014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:04.151533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:44:04.169216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:04.170610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:04.172228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:44:04.172228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:44:04.172228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:44:04.172228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:04.172228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 648/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:04.180387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:04.290110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e1937-98f1-4287-a109-b3724e21b67c None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:04.327041 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:04.327713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:04.327997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:04.328266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:04.328983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:04.329278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:04.330683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:04.342944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 631/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:04.348768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:04.384470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:04.478189 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b47ed-7795-4ebe-96f4-44cd0774f0b0 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:04.521722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:04.522967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:04.523381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:04.523847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:04.534556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:04.536862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 21 15:44:04.552831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:04.553947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:04.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 649/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:44:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:04.561328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:04.566104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:04.678494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220e1b2-5fce-4ce4-94c7-2f945a823c6d None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:04.715517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:04.716192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:04.716477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:04.716755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:04.717603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:04.717945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:04.719433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:04.737072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 632/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Jun 21 15:44:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:44:04.743407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:04.883406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:04.894496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19382b90-1ab2-44a3-b9a9-a10e5284abc6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:04.933346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7aa63-61e1-47d2-add2-0047b7f44cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:04.933980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7aa63-61e1-47d2-add2-0047b7f44cce None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:04.934267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7aa63-61e1-47d2-add2-0047b7f44cce None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:04.934583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7aa63-61e1-47d2-add2-0047b7f44cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:05.008036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7aa63-61e1-47d2-add2-0047b7f44cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:05.009493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7aa63-61e1-47d2-add2-0047b7f44cce None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:05.106949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:05.126412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 650/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Jun 21 15:44:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 21 15:44:05.135344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:05.207185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:05.253411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290ccad-ab43-46b6-ac70-98d38ade4c41 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:05.292380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:05.293177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:05.293458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:05.293728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:05.304578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:05.306141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 21 15:44:05.324033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:05.325024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:05.326263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 21 15:44:05.326263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 21 15:44:05.326263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 21 15:44:05.326263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:44:05.326263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 633/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:44:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:05.331302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7aa63-61e1-47d2-add2-0047b7f44cce None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:05.436258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7aa63-61e1-47d2-add2-0047b7f44cce None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:05.471761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:05.472340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:05.472626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:05.472899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:05.473618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:05.473954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:05.475329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:05.486815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 651/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Jun 21 15:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:05.493330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:05.605319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ede981-2e3b-4d2d-a7cd-42074e6f15ae None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:05.619750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:05.641781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:05.642598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:05.642876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:05.643148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] PATH_INFO: `/v3/projects/service` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:05.649952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:05.651393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 21 15:44:05.664852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:05.665894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: ERROR keystone.server.flask.application Jun 21 15:44:05.667116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 634/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Jun 21 15:44:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:05.672677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:05.745791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:05.801530 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020f10d-15a0-45bc-ba8b-51398cc465b9 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:05.835564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc118e-afe5-43f2-95ad-aba5f377f6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:05.836486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc118e-afe5-43f2-95ad-aba5f377f6a1 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:05.837045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc118e-afe5-43f2-95ad-aba5f377f6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:05.837322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc118e-afe5-43f2-95ad-aba5f377f6a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:05.838154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc118e-afe5-43f2-95ad-aba5f377f6a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:05.838464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc118e-afe5-43f2-95ad-aba5f377f6a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:05.839936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc118e-afe5-43f2-95ad-aba5f377f6a1 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:05.852671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 652/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Jun 21 15:44:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:05.862675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:05.984980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbb735-f160-4be6-8c9e-e02a2ca70dff None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:06.028601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f14ec2-1064-4f55-91ff-520bb1ffd597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:06.029162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14ec2-1064-4f55-91ff-520bb1ffd597 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:06.029439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14ec2-1064-4f55-91ff-520bb1ffd597 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:06.029693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f14ec2-1064-4f55-91ff-520bb1ffd597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:06.030663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f14ec2-1064-4f55-91ff-520bb1ffd597 None admin] RBAC: Adding query filter params (role.id=caaf25fef38b4e7e8601fe46da291ce8, scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f, user.id=16defb911ffa485da9164fa2674baba1) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:06.030960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f14ec2-1064-4f55-91ff-520bb1ffd597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:06.034313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f14ec2-1064-4f55-91ff-520bb1ffd597 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:06.042664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 635/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Jun 21 15:44:05 2022] GET /identity/v3/role_assignments?user.id=16defb911ffa485da9164fa2674baba1&scope.project.id=60408c00fa3d4698b251ca2a59ae8c5f&role.id=caaf25fef38b4e7e8601fe46da291ce8 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:08.123003 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7e5a-475b-40fe-88d9-558987538b58 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:08.123563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7e5a-475b-40fe-88d9-558987538b58 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:08.123563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef7e5a-475b-40fe-88d9-558987538b58 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:08.124930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 653/1288] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:44:08.130946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b07e-bc81-49af-93df-a9de51d6b3d9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:08.131407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b07e-bc81-49af-93df-a9de51d6b3d9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:08.131503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388b07e-bc81-49af-93df-a9de51d6b3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:08.150715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e388b07e-bc81-49af-93df-a9de51d6b3d9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:44:08.151927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e388b07e-bc81-49af-93df-a9de51d6b3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:44:08.185438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:08.231178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 636/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:44:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:44:08.254146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef7e5a-475b-40fe-88d9-558987538b58 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:08.383599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef7e5a-475b-40fe-88d9-558987538b58 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:08.395166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:08.423906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:08.426070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:08.426609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:08.427005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:08.428130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:08.431325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:08.447307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:08.448652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:08.449950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 654/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:44:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 21 15:44:08.455873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b07e-bc81-49af-93df-a9de51d6b3d9 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:08.572866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388b07e-bc81-49af-93df-a9de51d6b3d9 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:08.611695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c1837-8421-494f-9495-1c7837e5b082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:08.612359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1837-8421-494f-9495-1c7837e5b082 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:08.612644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1837-8421-494f-9495-1c7837e5b082 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:08.612919 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c1837-8421-494f-9495-1c7837e5b082 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:08.613628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c1837-8421-494f-9495-1c7837e5b082 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:08.613974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c1837-8421-494f-9495-1c7837e5b082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:08.615357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c1837-8421-494f-9495-1c7837e5b082 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:08.620934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 637/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Jun 21 15:44:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:08.629024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:08.734067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b7c25-84db-442d-90e2-5643c0b99159 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:08.769679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c08a4-1857-4a71-be1d-4d23559d9706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:08.770405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c08a4-1857-4a71-be1d-4d23559d9706 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:08.770695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c08a4-1857-4a71-be1d-4d23559d9706 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:08.770971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c08a4-1857-4a71-be1d-4d23559d9706 None admin] PATH_INFO: `/v3/services` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:08.771696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c08a4-1857-4a71-be1d-4d23559d9706 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:08.772007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c08a4-1857-4a71-be1d-4d23559d9706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:08.773457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c08a4-1857-4a71-be1d-4d23559d9706 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:08.778201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 655/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Jun 21 15:44:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:10.824184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0392fd-7370-4417-b052-87d61cfc7b78 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:10.824757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0392fd-7370-4417-b052-87d61cfc7b78 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:10.824757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0392fd-7370-4417-b052-87d61cfc7b78 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:10.826168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 638/1293] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:44:10.831629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3464-8c99-429b-8b70-76daef5a1c2e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:10.832092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3464-8c99-429b-8b70-76daef5a1c2e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:10.832185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a3464-8c99-429b-8b70-76daef5a1c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:10.850769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-cd0a3464-8c99-429b-8b70-76daef5a1c2e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:44:10.852308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cd0a3464-8c99-429b-8b70-76daef5a1c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:44:10.892933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:10.940202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 656/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:44:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:44:10.968910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0392fd-7370-4417-b052-87d61cfc7b78 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:11.006879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:11.096179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0392fd-7370-4417-b052-87d61cfc7b78 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:11.137192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14b8d2-1b50-4334-b57a-d43bb132ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:11.137899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14b8d2-1b50-4334-b57a-d43bb132ba6d None admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:11.138234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14b8d2-1b50-4334-b57a-d43bb132ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:11.138817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14b8d2-1b50-4334-b57a-d43bb132ba6d None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:11.139197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14b8d2-1b50-4334-b57a-d43bb132ba6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:11.140586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14b8d2-1b50-4334-b57a-d43bb132ba6d None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:11.155444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 639/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Jun 21 15:44:10 2022] POST /identity/v3/services => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:44:13.298436 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12284a5-6882-4dfe-8a99-3e6d59374562 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:13.299024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12284a5-6882-4dfe-8a99-3e6d59374562 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:13.299024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12284a5-6882-4dfe-8a99-3e6d59374562 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:13.300382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 657/1296] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:44:13.308439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2679e8-d363-4625-9c91-eb677a709e9f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:13.308932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2679e8-d363-4625-9c91-eb677a709e9f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:13.309051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2679e8-d363-4625-9c91-eb677a709e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:13.327816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1a2679e8-d363-4625-9c91-eb677a709e9f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:44:13.329003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1a2679e8-d363-4625-9c91-eb677a709e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:44:13.409482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 640/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:44:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:44:13.433849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12284a5-6882-4dfe-8a99-3e6d59374562 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:13.549704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12284a5-6882-4dfe-8a99-3e6d59374562 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:13.585231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:13.586113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:13.586408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:13.586680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:13.587325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:13.588738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:13.602259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:13.603615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:13.604983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 658/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:44:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:13.610524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2679e8-d363-4625-9c91-eb677a709e9f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:13.634972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:13.727302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2679e8-d363-4625-9c91-eb677a709e9f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:13.763648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:13.764325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:13.764605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:13.764883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:13.765587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:13.765928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:13.767305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:13.772467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 641/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Jun 21 15:44:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:13.778675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:13.812501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:13.893121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbcab2c-13bf-403c-beaf-269bb38c18d6 None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:13.932009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2ace1-a80e-4828-92d7-04ea98c56761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:13.932892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ace1-a80e-4828-92d7-04ea98c56761 None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:13.933203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ace1-a80e-4828-92d7-04ea98c56761 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:13.933548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ace1-a80e-4828-92d7-04ea98c56761 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:13.934551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2ace1-a80e-4828-92d7-04ea98c56761 None admin] RBAC: Adding query filter params (interface=public, service_id=57773a340b8f49d2b6a0a53dd64afde4, region_id=RegionOne) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:13.934967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2ace1-a80e-4828-92d7-04ea98c56761 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:13.936576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2ace1-a80e-4828-92d7-04ea98c56761 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:13.943087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 659/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Jun 21 15:44:13 2022] GET /identity/v3/endpoints?service_id=57773a340b8f49d2b6a0a53dd64afde4&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:13.949659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:14.057685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:14.080038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010da85-9f1b-4734-9d37-3e7cb20396e7 None admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:14.117862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a3716-2983-4209-b73c-233756b30bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:14.118320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a3716-2983-4209-b73c-233756b30bc9 None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:14.118608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a3716-2983-4209-b73c-233756b30bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:14.118877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a3716-2983-4209-b73c-233756b30bc9 None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:14.119701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a3716-2983-4209-b73c-233756b30bc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:14.121091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a3716-2983-4209-b73c-233756b30bc9 None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:14.128212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 642/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Jun 21 15:44:13 2022] GET /identity/v3/services => generated 2191 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:44:16.177370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1696d-7eb4-411a-b3a1-82e2a2a91532 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:16.178338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1696d-7eb4-411a-b3a1-82e2a2a91532 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:16.178338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1696d-7eb4-411a-b3a1-82e2a2a91532 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:16.180094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 660/1302] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:44:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:44:16.185716 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f344762-513b-4dd2-9240-688a4203cc7d None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:16.186200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f344762-513b-4dd2-9240-688a4203cc7d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:16.186302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f344762-513b-4dd2-9240-688a4203cc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:16.203595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3f344762-513b-4dd2-9240-688a4203cc7d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:44:16.204764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3f344762-513b-4dd2-9240-688a4203cc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:44:16.283699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 643/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:44:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 21 15:44:16.308232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1696d-7eb4-411a-b3a1-82e2a2a91532 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:16.426992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:16.434517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1696d-7eb4-411a-b3a1-82e2a2a91532 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:16.473191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:16.474024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:16.474301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:16.474567 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] PATH_INFO: `/v3/services/placement` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:16.475216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:16.476924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 21 15:44:16.489919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 21 15:44:16.491128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.get_or_create( Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application with Lock( Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self._enter() Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application return self.creator() Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application created_value = creator( Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: ERROR keystone.server.flask.application Jun 21 15:44:16.492360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 661/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Jun 21 15:44:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 21 15:44:16.498318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f344762-513b-4dd2-9240-688a4203cc7d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:16.586788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:16.614594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f344762-513b-4dd2-9240-688a4203cc7d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:16.666976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cba31f-d7b0-48c7-a3c8-942be232b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:16.667649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cba31f-d7b0-48c7-a3c8-942be232b6ad None admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:16.667934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cba31f-d7b0-48c7-a3c8-942be232b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:16.668347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cba31f-d7b0-48c7-a3c8-942be232b6ad None admin] PATH_INFO: `/v3/services` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:16.669093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cba31f-d7b0-48c7-a3c8-942be232b6ad None admin] RBAC: Adding query filter params (name=placement) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:44:16.669404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cba31f-d7b0-48c7-a3c8-942be232b6ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:16.670842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cba31f-d7b0-48c7-a3c8-942be232b6ad None admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:16.677406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 644/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Jun 21 15:44:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:44:16.684454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:44:16.796864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c291ba-f3cb-41fd-a37a-047a22a4f6ac None admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:44:16.848926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1290ea-b43e-42eb-a989-e6ede1632db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:44:16.850022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1290ea-b43e-42eb-a989-e6ede1632db9 None admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:44:16.850501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1290ea-b43e-42eb-a989-e6ede1632db9 None admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:44:16.850987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1290ea-b43e-42eb-a989-e6ede1632db9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:44:16.851764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1290ea-b43e-42eb-a989-e6ede1632db9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:44:16.853174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1290ea-b43e-42eb-a989-e6ede1632db9 None admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:44:16.878270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 662/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Jun 21 15:44:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:44:26.673617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=30 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:44:26.871393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:21.425693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a57f1-bcf6-408b-b3aa-041b95a7d2c0 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:21.426317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a57f1-bcf6-408b-b3aa-041b95a7d2c0 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:21.426317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a57f1-bcf6-408b-b3aa-041b95a7d2c0 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:21.433232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8ca8f-f181-48d6-8b46-aa4e61e3d573 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:21.434002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8ca8f-f181-48d6-8b46-aa4e61e3d573 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:21.434002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8ca8f-f181-48d6-8b46-aa4e61e3d573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:21.437303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 645/1307] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:21 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:21.577529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-9ba8ca8f-f181-48d6-8b46-aa4e61e3d573 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:21.578770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9ba8ca8f-f181-48d6-8b46-aa4e61e3d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:21.791614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 663/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:45:21.841974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e144697-43d2-466b-84b7-653fdd5fff29 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:21.842772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e144697-43d2-466b-84b7-653fdd5fff29 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:21.843389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e144697-43d2-466b-84b7-653fdd5fff29 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:21.847666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 646/1309] 10.222.0.52 () {54 vars in 964 bytes} [Tue Jun 21 15:45:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:21.854266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802030a2-cbc1-475a-959b-5c0213d374e5 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:21.855090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802030a2-cbc1-475a-959b-5c0213d374e5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:21.855984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802030a2-cbc1-475a-959b-5c0213d374e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:22.038926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-802030a2-cbc1-475a-959b-5c0213d374e5 None None] MFA Rules not processed for user `6efa4bed025741d492158b5c6e45c7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:22.041452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-802030a2-cbc1-475a-959b-5c0213d374e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:22.091546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:22.219206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 664/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Jun 21 15:45:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:22.228390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e144697-43d2-466b-84b7-653fdd5fff29 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:22.230245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3e144697-43d2-466b-84b7-653fdd5fff29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:22.446084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e144697-43d2-466b-84b7-653fdd5fff29 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:22.555100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb0c55-b0df-4685-900f-499a4101a7de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:22.555764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0c55-b0df-4685-900f-499a4101a7de service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:22.556174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0c55-b0df-4685-900f-499a4101a7de service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:22.556579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb0c55-b0df-4685-900f-499a4101a7de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:22.558287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-96bb0c55-b0df-4685-900f-499a4101a7de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:22.638717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:22.722901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb0c55-b0df-4685-900f-499a4101a7de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:22.724402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb0c55-b0df-4685-900f-499a4101a7de service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:22.748524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 647/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:25.179028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fe49f-43a0-4a9e-97cb-6f3ff1cc4fcd None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:25.179707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fe49f-43a0-4a9e-97cb-6f3ff1cc4fcd None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:25.179707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fe49f-43a0-4a9e-97cb-6f3ff1cc4fcd None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:25.181026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 665/1312] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:25.186981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00790b-1386-4329-b0ea-c06fafac2da9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:25.187600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00790b-1386-4329-b0ea-c06fafac2da9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:25.187600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00790b-1386-4329-b0ea-c06fafac2da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:25.280811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-5e00790b-1386-4329-b0ea-c06fafac2da9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:25.282221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5e00790b-1386-4329-b0ea-c06fafac2da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:25.388243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 648/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:25.427567 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fe49f-43a0-4a9e-97cb-6f3ff1cc4fcd None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:25.428719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d71fe49f-43a0-4a9e-97cb-6f3ff1cc4fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:25.547666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fe49f-43a0-4a9e-97cb-6f3ff1cc4fcd None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:25.624630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:25.627391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab15df-c5b3-4304-ba4c-d85fdd3207c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:25.628333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab15df-c5b3-4304-ba4c-d85fdd3207c4 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:25.628882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab15df-c5b3-4304-ba4c-d85fdd3207c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:25.629294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab15df-c5b3-4304-ba4c-d85fdd3207c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:25.631608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e1ab15df-c5b3-4304-ba4c-d85fdd3207c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:25.757358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab15df-c5b3-4304-ba4c-d85fdd3207c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:25.758749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ab15df-c5b3-4304-ba4c-d85fdd3207c4 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:25.778534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 666/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:45:28.222701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0813-8576-42e4-af23-d44feceb99b2 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:28.223264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0813-8576-42e4-af23-d44feceb99b2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:28.223264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0813-8576-42e4-af23-d44feceb99b2 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:28.224694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 649/1315] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:28.230135 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577e95f-f6f5-4fa8-96a9-d1010d1a3fef None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:28.230623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577e95f-f6f5-4fa8-96a9-d1010d1a3fef None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:28.230759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577e95f-f6f5-4fa8-96a9-d1010d1a3fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:28.301650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5577e95f-f6f5-4fa8-96a9-d1010d1a3fef None None] MFA Rules not processed for user `4d94401bc90146c29a9e891bdf7a2ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:28.302914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5577e95f-f6f5-4fa8-96a9-d1010d1a3fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:28.359461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:28.408894 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 667/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:28.463632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d8e1b-98a1-4474-8ddb-4e1a36999ceb None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:28.463920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d8e1b-98a1-4474-8ddb-4e1a36999ceb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:28.464197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d8e1b-98a1-4474-8ddb-4e1a36999ceb None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:28.465665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 650/1317] 10.222.0.52 () {54 vars in 964 bytes} [Tue Jun 21 15:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:28.472248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710f51c-bf2f-4e41-b7a0-bacc9621f53e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:28.472536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710f51c-bf2f-4e41-b7a0-bacc9621f53e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:28.472869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710f51c-bf2f-4e41-b7a0-bacc9621f53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:28.554377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d710f51c-bf2f-4e41-b7a0-bacc9621f53e None None] MFA Rules not processed for user `6efa4bed025741d492158b5c6e45c7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:28.555604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d710f51c-bf2f-4e41-b7a0-bacc9621f53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:28.664987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 668/1318] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Jun 21 15:45:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:28.674237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d8e1b-98a1-4474-8ddb-4e1a36999ceb None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:28.675799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-161d8e1b-98a1-4474-8ddb-4e1a36999ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:28.685741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:28.784467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d8e1b-98a1-4474-8ddb-4e1a36999ceb None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:28.861720 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd86cb-a6e9-4907-bc21-deeac0abaefb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:28.862405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd86cb-a6e9-4907-bc21-deeac0abaefb service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:28.862694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd86cb-a6e9-4907-bc21-deeac0abaefb service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:28.862973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd86cb-a6e9-4907-bc21-deeac0abaefb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:28.864647 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-04dd86cb-a6e9-4907-bc21-deeac0abaefb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:28.935631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:28.992150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd86cb-a6e9-4907-bc21-deeac0abaefb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:28.993621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd86cb-a6e9-4907-bc21-deeac0abaefb service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:29.015863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 651/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:28 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:31.952395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1284186-fc22-4a04-9a2d-eb7d359bac1c None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:31.953190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1284186-fc22-4a04-9a2d-eb7d359bac1c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:31.953190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1284186-fc22-4a04-9a2d-eb7d359bac1c None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:31.954517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 669/1320] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:31.960101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc77cb-8caa-4fa4-a6b1-eff7b1ea43d2 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:31.960534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc77cb-8caa-4fa4-a6b1-eff7b1ea43d2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:31.960625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc77cb-8caa-4fa4-a6b1-eff7b1ea43d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:32.030763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3bfc77cb-8caa-4fa4-a6b1-eff7b1ea43d2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:32.032033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3bfc77cb-8caa-4fa4-a6b1-eff7b1ea43d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:32.143646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 652/1321] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:32.196665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1284186-fc22-4a04-9a2d-eb7d359bac1c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:32.199121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c1284186-fc22-4a04-9a2d-eb7d359bac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:32.255166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:32.311172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1284186-fc22-4a04-9a2d-eb7d359bac1c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:32.400507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe813cb-a9f8-49a4-981a-0e11769df57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:32.401425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe813cb-a9f8-49a4-981a-0e11769df57d service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:32.401850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe813cb-a9f8-49a4-981a-0e11769df57d service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:32.402300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe813cb-a9f8-49a4-981a-0e11769df57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:32.404957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cbe813cb-a9f8-49a4-981a-0e11769df57d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:32.515622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:32.531908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe813cb-a9f8-49a4-981a-0e11769df57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:32.533377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe813cb-a9f8-49a4-981a-0e11769df57d service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:32.553286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 670/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:35.338947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dd763-cde7-4695-9691-63ea6ee28942 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:35.339530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dd763-cde7-4695-9691-63ea6ee28942 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:35.339530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dd763-cde7-4695-9691-63ea6ee28942 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:35.340904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 653/1323] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:35.346670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e747-9648-42c9-bb39-4e57835540bf None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:35.347131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e747-9648-42c9-bb39-4e57835540bf None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:35.347240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e747-9648-42c9-bb39-4e57835540bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:35.415701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3c82e747-9648-42c9-bb39-4e57835540bf None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:35.416927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3c82e747-9648-42c9-bb39-4e57835540bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:35.548172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 671/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:35.586818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dd763-cde7-4695-9691-63ea6ee28942 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:35.589313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-cc6dd763-cde7-4695-9691-63ea6ee28942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:35.612621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:35.708974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dd763-cde7-4695-9691-63ea6ee28942 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:35.786657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd930a-94bb-4758-9210-e51dad116c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:35.787308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd930a-94bb-4758-9210-e51dad116c3d service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:35.787605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd930a-94bb-4758-9210-e51dad116c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:35.787874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd930a-94bb-4758-9210-e51dad116c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:35.789555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-03bd930a-94bb-4758-9210-e51dad116c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:35.873689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:35.912904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd930a-94bb-4758-9210-e51dad116c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:35.914312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd930a-94bb-4758-9210-e51dad116c3d service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:35.933712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 654/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:38.051492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71bc62-ee91-4d5d-b95c-259db09068ad None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:38.052079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71bc62-ee91-4d5d-b95c-259db09068ad None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:38.052079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71bc62-ee91-4d5d-b95c-259db09068ad None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:38.053327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 672/1326] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:38.059350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a40d6-eb7c-4933-849a-2bef4d0c9047 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:38.060048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a40d6-eb7c-4933-849a-2bef4d0c9047 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:38.060048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a40d6-eb7c-4933-849a-2bef4d0c9047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:38.149595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d73a40d6-eb7c-4933-849a-2bef4d0c9047 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:38.150862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d73a40d6-eb7c-4933-849a-2bef4d0c9047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:38.265592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 655/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:45:38.301924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71bc62-ee91-4d5d-b95c-259db09068ad None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:38.303502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-fd71bc62-ee91-4d5d-b95c-259db09068ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:38.372310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:38.423724 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71bc62-ee91-4d5d-b95c-259db09068ad None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:38.501299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f300e48-6fb1-4b6f-9e87-98dd1a3c159e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:38.501992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f300e48-6fb1-4b6f-9e87-98dd1a3c159e service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:38.502242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f300e48-6fb1-4b6f-9e87-98dd1a3c159e service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:38.502533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f300e48-6fb1-4b6f-9e87-98dd1a3c159e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:38.504195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2f300e48-6fb1-4b6f-9e87-98dd1a3c159e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:38.629396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:38.632316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f300e48-6fb1-4b6f-9e87-98dd1a3c159e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:38.634196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f300e48-6fb1-4b6f-9e87-98dd1a3c159e service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:38.653330 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 673/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:40.747901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a16ee-abae-49bf-8ad8-038676be6155 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:40.748414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a16ee-abae-49bf-8ad8-038676be6155 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:40.748534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a16ee-abae-49bf-8ad8-038676be6155 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:40.750238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 656/1329] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:40.755939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fc6af-e847-4545-b770-99b8a04a8daf None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:40.756390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fc6af-e847-4545-b770-99b8a04a8daf None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:40.756479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fc6af-e847-4545-b770-99b8a04a8daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:40.827501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3d2fc6af-e847-4545-b770-99b8a04a8daf None None] MFA Rules not processed for user `4d94401bc90146c29a9e891bdf7a2ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:40.828965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3d2fc6af-e847-4545-b770-99b8a04a8daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:40.936513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 674/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:40.974592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a16ee-abae-49bf-8ad8-038676be6155 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:40.976230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f31a16ee-abae-49bf-8ad8-038676be6155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:41.008863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:41.087846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a16ee-abae-49bf-8ad8-038676be6155 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:41.168272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e85b5-5e98-4d88-ba2d-df05e66f6180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:41.168933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e85b5-5e98-4d88-ba2d-df05e66f6180 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:41.169178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e85b5-5e98-4d88-ba2d-df05e66f6180 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:41.169479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e85b5-5e98-4d88-ba2d-df05e66f6180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:41.171216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3b9e85b5-5e98-4d88-ba2d-df05e66f6180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:41.270508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:41.307571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e85b5-5e98-4d88-ba2d-df05e66f6180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:41.309034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e85b5-5e98-4d88-ba2d-df05e66f6180 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:41.329729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 657/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:43.626769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe803-aed9-450b-afe5-2e259bdb72e9 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:43.627313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe803-aed9-450b-afe5-2e259bdb72e9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:43.627313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe803-aed9-450b-afe5-2e259bdb72e9 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:43.628735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 675/1332] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:43.634377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9164cdd-ec3f-4616-b549-c0ce28f7d209 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:43.634850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9164cdd-ec3f-4616-b549-c0ce28f7d209 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:43.634956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9164cdd-ec3f-4616-b549-c0ce28f7d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:43.702351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b9164cdd-ec3f-4616-b549-c0ce28f7d209 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:43.703574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b9164cdd-ec3f-4616-b549-c0ce28f7d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:43.804430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 658/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:43.840080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fe803-aed9-450b-afe5-2e259bdb72e9 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:43.841654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e57fe803-aed9-450b-afe5-2e259bdb72e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:43.916707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:43.953495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fe803-aed9-450b-afe5-2e259bdb72e9 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:44.036960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58fc6b-d66f-437c-84e8-4b48997efc9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:44.037606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58fc6b-d66f-437c-84e8-4b48997efc9c service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:44.037914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58fc6b-d66f-437c-84e8-4b48997efc9c service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:44.038188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58fc6b-d66f-437c-84e8-4b48997efc9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:44.039869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ff58fc6b-d66f-437c-84e8-4b48997efc9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:44.168347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58fc6b-d66f-437c-84e8-4b48997efc9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:44.169700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58fc6b-d66f-437c-84e8-4b48997efc9c service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:44.180439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:44.189748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 676/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:46.307717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f494bb-7ec2-441a-9909-97b022ce45c6 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:46.308303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f494bb-7ec2-441a-9909-97b022ce45c6 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:46.308303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f494bb-7ec2-441a-9909-97b022ce45c6 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:46.309631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 659/1335] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:45:46.315355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4e14e-14b1-44e9-88ae-1a9ae664eff3 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:46.315773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4e14e-14b1-44e9-88ae-1a9ae664eff3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:46.315884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4e14e-14b1-44e9-88ae-1a9ae664eff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:46.382320 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-aac4e14e-14b1-44e9-88ae-1a9ae664eff3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:46.383529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-aac4e14e-14b1-44e9-88ae-1a9ae664eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:46.497660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 677/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:46.537826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f494bb-7ec2-441a-9909-97b022ce45c6 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:46.539452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-75f494bb-7ec2-441a-9909-97b022ce45c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:46.582230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:46.652225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f494bb-7ec2-441a-9909-97b022ce45c6 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:46.730139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d654c4-64e7-44ee-a005-a777201c70fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:46.730771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d654c4-64e7-44ee-a005-a777201c70fb service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:46.731051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d654c4-64e7-44ee-a005-a777201c70fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:46.731325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d654c4-64e7-44ee-a005-a777201c70fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:46.733012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-29d654c4-64e7-44ee-a005-a777201c70fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:46.822037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:46.853867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d654c4-64e7-44ee-a005-a777201c70fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:46.855332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d654c4-64e7-44ee-a005-a777201c70fb service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:46.875816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 660/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:49.263413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153daf2-5a0a-4a88-a804-f6ac24752a35 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:49.263928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153daf2-5a0a-4a88-a804-f6ac24752a35 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:49.263928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153daf2-5a0a-4a88-a804-f6ac24752a35 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:49.265620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 678/1338] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:49.271202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874ff0-dad7-4d0e-a95e-8cb25eaf2e3f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:49.271677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874ff0-dad7-4d0e-a95e-8cb25eaf2e3f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:49.271769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874ff0-dad7-4d0e-a95e-8cb25eaf2e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:49.339088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-95874ff0-dad7-4d0e-a95e-8cb25eaf2e3f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:49.340331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-95874ff0-dad7-4d0e-a95e-8cb25eaf2e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:49.441056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 661/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:49.477803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153daf2-5a0a-4a88-a804-f6ac24752a35 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:49.479360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7153daf2-5a0a-4a88-a804-f6ac24752a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:49.580711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:49.614648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153daf2-5a0a-4a88-a804-f6ac24752a35 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:49.692828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de520cb6-f726-46c5-adb7-65dc254618ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:49.693481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de520cb6-f726-46c5-adb7-65dc254618ad service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:49.693757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de520cb6-f726-46c5-adb7-65dc254618ad service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:49.694060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de520cb6-f726-46c5-adb7-65dc254618ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:49.695708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-de520cb6-f726-46c5-adb7-65dc254618ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:49.814059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de520cb6-f726-46c5-adb7-65dc254618ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:49.815512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de520cb6-f726-46c5-adb7-65dc254618ad service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:49.835797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 679/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:54.163007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00a07-3625-4bcf-aade-7a93a750e49b None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:54.163617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00a07-3625-4bcf-aade-7a93a750e49b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:54.163617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf00a07-3625-4bcf-aade-7a93a750e49b None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:54.164944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 662/1341] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:45:54.170545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d086b-4b79-48a2-9c54-94d02305ac73 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:54.171719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d086b-4b79-48a2-9c54-94d02305ac73 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:54.171719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d086b-4b79-48a2-9c54-94d02305ac73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:54.179169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:54.252810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-9c4d086b-4b79-48a2-9c54-94d02305ac73 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:54.254038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9c4d086b-4b79-48a2-9c54-94d02305ac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:54.376723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 680/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:54.415598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00a07-3625-4bcf-aade-7a93a750e49b None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:54.418079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-cbf00a07-3625-4bcf-aade-7a93a750e49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:54.476590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:45:54.533202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf00a07-3625-4bcf-aade-7a93a750e49b None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:54.536807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cad82-c7c7-422f-b9ee-f8ffa8acb3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:54.537464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cad82-c7c7-422f-b9ee-f8ffa8acb3cd service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:54.537857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cad82-c7c7-422f-b9ee-f8ffa8acb3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:54.538049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cad82-c7c7-422f-b9ee-f8ffa8acb3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:54.548743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cad82-c7c7-422f-b9ee-f8ffa8acb3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:54.550357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59cad82-c7c7-422f-b9ee-f8ffa8acb3cd service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:54.563427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 663/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:45:57.291140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb32ea6-dc3f-4669-bb33-89872c051c17 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:57.291764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb32ea6-dc3f-4669-bb33-89872c051c17 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:57.291764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb32ea6-dc3f-4669-bb33-89872c051c17 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:57.293067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 681/1344] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:45:57.298576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c05bd5-326c-4d62-9253-da5433b59b5f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:57.299068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c05bd5-326c-4d62-9253-da5433b59b5f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:57.299174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c05bd5-326c-4d62-9253-da5433b59b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:57.328859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-08c05bd5-326c-4d62-9253-da5433b59b5f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:45:57.330195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-08c05bd5-326c-4d62-9253-da5433b59b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:45:57.421207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 664/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:45:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:45:57.460162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb32ea6-dc3f-4669-bb33-89872c051c17 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:45:57.473357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb32ea6-dc3f-4669-bb33-89872c051c17 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:45:57.476937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b6167-fd7e-437e-9430-8d493e882815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:45:57.477708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6167-fd7e-437e-9430-8d493e882815 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:45:57.478061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6167-fd7e-437e-9430-8d493e882815 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:45:57.478335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6167-fd7e-437e-9430-8d493e882815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:45:57.489728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b6167-fd7e-437e-9430-8d493e882815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:45:57.491449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b6167-fd7e-437e-9430-8d493e882815 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:45:57.503660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 682/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:45:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:01.948628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619ed30-b145-4c9a-89d8-aca6bd8da8ec None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:01.949182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619ed30-b145-4c9a-89d8-aca6bd8da8ec None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:01.949182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619ed30-b145-4c9a-89d8-aca6bd8da8ec None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:01.950807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 665/1347] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:01.956190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7742-0e82-423f-a113-db47f856c6df None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:01.956648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7742-0e82-423f-a113-db47f856c6df None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:01.956873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7742-0e82-423f-a113-db47f856c6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:01.974690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-804e7742-0e82-423f-a113-db47f856c6df None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:01.975901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-804e7742-0e82-423f-a113-db47f856c6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:01.984163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 683/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:02.022992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619ed30-b145-4c9a-89d8-aca6bd8da8ec None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:02.040081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619ed30-b145-4c9a-89d8-aca6bd8da8ec None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:02.043680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3cf97-6c17-432c-a5ad-f5d39deaa30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:02.044404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3cf97-6c17-432c-a5ad-f5d39deaa30b service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:02.044821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3cf97-6c17-432c-a5ad-f5d39deaa30b service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:02.045107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3cf97-6c17-432c-a5ad-f5d39deaa30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:02.056775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3cf97-6c17-432c-a5ad-f5d39deaa30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:02.058369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3cf97-6c17-432c-a5ad-f5d39deaa30b service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:02.069492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 666/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:07.326923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ecc71-22e9-45e0-abb9-daedde2d1b4e None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:07.327828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ecc71-22e9-45e0-abb9-daedde2d1b4e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:07.328409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ecc71-22e9-45e0-abb9-daedde2d1b4e None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:07.330339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 684/1350] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:07.336417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40af015-9670-4bea-824f-cad31b36b963 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:07.336869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40af015-9670-4bea-824f-cad31b36b963 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:07.337203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40af015-9670-4bea-824f-cad31b36b963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:07.487346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e40af015-9670-4bea-824f-cad31b36b963 None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:07.488886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e40af015-9670-4bea-824f-cad31b36b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:07.522772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:46:07.576022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 667/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:46:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:11.844905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8df99b-6306-47da-a359-c0be8c106097 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:11.845480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8df99b-6306-47da-a359-c0be8c106097 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:11.845480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8df99b-6306-47da-a359-c0be8c106097 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:11.846957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 685/1352] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:11.852419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ab8a3-cdb2-4e13-9663-fff9824749fb None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:11.852948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ab8a3-cdb2-4e13-9663-fff9824749fb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:11.852948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ab8a3-cdb2-4e13-9663-fff9824749fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:11.874070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f63ab8a3-cdb2-4e13-9663-fff9824749fb None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:11.875666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f63ab8a3-cdb2-4e13-9663-fff9824749fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:11.887731 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 668/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:46:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:11.969381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=72 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:46:13.537945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6767f-00f4-45a2-932a-bceabe416476 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:13.538573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6767f-00f4-45a2-932a-bceabe416476 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:13.538573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6767f-00f4-45a2-932a-bceabe416476 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:13.540084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 686/1354] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:13.544844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75460413-de3a-41fa-b771-3a6942443f9f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:13.545250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75460413-de3a-41fa-b771-3a6942443f9f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:13.545402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75460413-de3a-41fa-b771-3a6942443f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:13.564631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-75460413-de3a-41fa-b771-3a6942443f9f None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:13.565866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-75460413-de3a-41fa-b771-3a6942443f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:13.575133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 669/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:46:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:17.314368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308320-ad92-4c44-810a-15388e3c4080 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:17.314931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308320-ad92-4c44-810a-15388e3c4080 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:17.314931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308320-ad92-4c44-810a-15388e3c4080 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:17.316797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 687/1356] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:17.322819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5d69e-ebda-4d05-b9d5-4b49bb301568 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:17.323292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5d69e-ebda-4d05-b9d5-4b49bb301568 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:17.323387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5d69e-ebda-4d05-b9d5-4b49bb301568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:17.346367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-44a5d69e-ebda-4d05-b9d5-4b49bb301568 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:17.347555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-44a5d69e-ebda-4d05-b9d5-4b49bb301568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:17.357439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 670/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:17.416619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cc6d-3aa7-4b20-8885-6de6902a102f None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:17.417174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cc6d-3aa7-4b20-8885-6de6902a102f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:17.417753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cc6d-3aa7-4b20-8885-6de6902a102f None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:17.420129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 688/1358] 10.222.0.52 () {54 vars in 961 bytes} [Tue Jun 21 15:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:17.430962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81be159-0f9c-4fd0-b309-c186ffa5e8fd None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:17.431596 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81be159-0f9c-4fd0-b309-c186ffa5e8fd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:17.432019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81be159-0f9c-4fd0-b309-c186ffa5e8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:17.510756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d81be159-0f9c-4fd0-b309-c186ffa5e8fd None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:17.512002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d81be159-0f9c-4fd0-b309-c186ffa5e8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:17.615257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 671/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Jun 21 15:46:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:46:17.624711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9cc6d-3aa7-4b20-8885-6de6902a102f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:17.640188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9cc6d-3aa7-4b20-8885-6de6902a102f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:17.646617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eedee1-51b2-4daf-844f-4fc2f46228d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:17.647282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eedee1-51b2-4daf-844f-4fc2f46228d7 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:17.647564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eedee1-51b2-4daf-844f-4fc2f46228d7 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:17.647839 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eedee1-51b2-4daf-844f-4fc2f46228d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:17.657306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eedee1-51b2-4daf-844f-4fc2f46228d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:17.659273 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eedee1-51b2-4daf-844f-4fc2f46228d7 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:17.670247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 689/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:19.922069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a8457-35d9-466a-9abf-2daa7315d050 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:19.922679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a8457-35d9-466a-9abf-2daa7315d050 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:19.922759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a8457-35d9-466a-9abf-2daa7315d050 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:19.924334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 672/1361] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:19.929726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f169823-4b5d-4c4d-886a-e877ad0d38fc None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:19.930192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f169823-4b5d-4c4d-886a-e877ad0d38fc None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:19.930271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f169823-4b5d-4c4d-886a-e877ad0d38fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:19.952712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-4f169823-4b5d-4c4d-886a-e877ad0d38fc None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:19.954222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4f169823-4b5d-4c4d-886a-e877ad0d38fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:19.969843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 690/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:20.008976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a8457-35d9-466a-9abf-2daa7315d050 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:20.024715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a8457-35d9-466a-9abf-2daa7315d050 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:20.030337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2cfe7-125c-43e6-8c51-8bc2e81e4af0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:20.031067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cfe7-125c-43e6-8c51-8bc2e81e4af0 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:20.031472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cfe7-125c-43e6-8c51-8bc2e81e4af0 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:20.031819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cfe7-125c-43e6-8c51-8bc2e81e4af0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:20.041447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2cfe7-125c-43e6-8c51-8bc2e81e4af0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:20.043184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2cfe7-125c-43e6-8c51-8bc2e81e4af0 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:20.054225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 673/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:22.483658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e099e-f972-4f49-8428-5e54e306e26c None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:22.484194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e099e-f972-4f49-8428-5e54e306e26c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:22.484194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e099e-f972-4f49-8428-5e54e306e26c None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:22.485730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 691/1364] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:22.491101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d443b5-701d-48dd-bfe0-149f08db3d90 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:22.491549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d443b5-701d-48dd-bfe0-149f08db3d90 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:22.491631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d443b5-701d-48dd-bfe0-149f08db3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:22.511914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c9d443b5-701d-48dd-bfe0-149f08db3d90 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:22.513136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c9d443b5-701d-48dd-bfe0-149f08db3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:22.522245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 674/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:22.567513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1030e-9057-4e37-b370-99744b7d0b61 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:22.567803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1030e-9057-4e37-b370-99744b7d0b61 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:22.568053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1030e-9057-4e37-b370-99744b7d0b61 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:22.569477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 692/1366] 10.222.0.52 () {54 vars in 961 bytes} [Tue Jun 21 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:22.574846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b233-a791-45bb-852e-bd19be25cce7 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:22.575134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b233-a791-45bb-852e-bd19be25cce7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:22.575408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4b233-a791-45bb-852e-bd19be25cce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:22.594616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-42a4b233-a791-45bb-852e-bd19be25cce7 None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:22.595829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-42a4b233-a791-45bb-852e-bd19be25cce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:22.605292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 675/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Jun 21 15:46:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:22.613872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1030e-9057-4e37-b370-99744b7d0b61 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:22.629180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1030e-9057-4e37-b370-99744b7d0b61 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:22.634749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8eed3-3660-4182-8161-3a7f98a350a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:22.635415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8eed3-3660-4182-8161-3a7f98a350a7 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:22.635680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8eed3-3660-4182-8161-3a7f98a350a7 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:22.635954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8eed3-3660-4182-8161-3a7f98a350a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:22.644765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8eed3-3660-4182-8161-3a7f98a350a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:22.646417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8eed3-3660-4182-8161-3a7f98a350a7 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:22.655934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 693/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:24.763751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cf616-566d-46f6-9073-f24228ed0cb0 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:24.764359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cf616-566d-46f6-9073-f24228ed0cb0 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:24.764359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cf616-566d-46f6-9073-f24228ed0cb0 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:24.765702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 676/1369] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:24.772009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08bd6-98fb-4a7e-8479-845eca9e6246 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:24.772841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08bd6-98fb-4a7e-8479-845eca9e6246 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:24.773305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e08bd6-98fb-4a7e-8479-845eca9e6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:24.792068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-97e08bd6-98fb-4a7e-8479-845eca9e6246 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:24.793546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-97e08bd6-98fb-4a7e-8479-845eca9e6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:24.802289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 694/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:24.847705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cf616-566d-46f6-9073-f24228ed0cb0 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:24.863676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cf616-566d-46f6-9073-f24228ed0cb0 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:24.869602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839fbf4-131e-4470-9a7b-1369f4c29ac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:24.870512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839fbf4-131e-4470-9a7b-1369f4c29ac0 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:24.870980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839fbf4-131e-4470-9a7b-1369f4c29ac0 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:24.871440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839fbf4-131e-4470-9a7b-1369f4c29ac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:24.881257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839fbf4-131e-4470-9a7b-1369f4c29ac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:24.883166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839fbf4-131e-4470-9a7b-1369f4c29ac0 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:24.898140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 677/1371] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:25.453619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198c052-5e40-4883-8265-a2c3329974b1 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:25.453965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198c052-5e40-4883-8265-a2c3329974b1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:25.454509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b198c052-5e40-4883-8265-a2c3329974b1 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:25.456498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 695/1372] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:25.461921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e57c9-0e88-4a5c-b443-099382792c2c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:25.462217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e57c9-0e88-4a5c-b443-099382792c2c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:25.462497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e57c9-0e88-4a5c-b443-099382792c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:25.481946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-340e57c9-0e88-4a5c-b443-099382792c2c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:25.483172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-340e57c9-0e88-4a5c-b443-099382792c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:25.493112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 678/1373] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:25.843329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a8fb9-0297-43a8-88d0-31e3004acf50 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:25.843329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a8fb9-0297-43a8-88d0-31e3004acf50 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:25.843329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a8fb9-0297-43a8-88d0-31e3004acf50 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:25.843329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 696/1374] 10.222.0.52 () {54 vars in 963 bytes} [Tue Jun 21 15:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:25.849765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7297d-1c6f-4c77-ad4c-f5130c9b2576 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:25.849765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7297d-1c6f-4c77-ad4c-f5130c9b2576 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:25.849765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7297d-1c6f-4c77-ad4c-f5130c9b2576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:25.857155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:46:25.905520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-bed7297d-1c6f-4c77-ad4c-f5130c9b2576 None None] MFA Rules not processed for user `ed0934c0b1a64dfca3431dc32441e2db`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:25.907671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bed7297d-1c6f-4c77-ad4c-f5130c9b2576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:25.977609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 679/1375] 10.222.0.52 () {58 vars in 1060 bytes} [Tue Jun 21 15:46:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:25.985833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a8fb9-0297-43a8-88d0-31e3004acf50 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:25.997716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a8fb9-0297-43a8-88d0-31e3004acf50 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:26.002151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c2c88-8c6e-4b7e-9943-f71eb102ae61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:26.002964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c2c88-8c6e-4b7e-9943-f71eb102ae61 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:26.003347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c2c88-8c6e-4b7e-9943-f71eb102ae61 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:26.003723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c2c88-8c6e-4b7e-9943-f71eb102ae61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:26.014595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c2c88-8c6e-4b7e-9943-f71eb102ae61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:26.016532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c2c88-8c6e-4b7e-9943-f71eb102ae61 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:26.027574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 697/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:46:27.057322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10987-31ad-4582-8ebd-2ce46407c4ce None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:27.057924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10987-31ad-4582-8ebd-2ce46407c4ce None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:27.057924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10987-31ad-4582-8ebd-2ce46407c4ce None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:27.059291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 680/1377] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:27.064860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a3e7d-e7fe-4faa-a91a-469a2a910ef7 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:27.065411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a3e7d-e7fe-4faa-a91a-469a2a910ef7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:27.065411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a3e7d-e7fe-4faa-a91a-469a2a910ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:27.089509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ee4a3e7d-e7fe-4faa-a91a-469a2a910ef7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:27.090753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ee4a3e7d-e7fe-4faa-a91a-469a2a910ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:27.099245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 698/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:27.146147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea10987-31ad-4582-8ebd-2ce46407c4ce None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:27.162709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea10987-31ad-4582-8ebd-2ce46407c4ce None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:27.175690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391eb6d3-1e82-4099-bb97-631ecca4a430 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:27.176335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eb6d3-1e82-4099-bb97-631ecca4a430 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:27.176763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eb6d3-1e82-4099-bb97-631ecca4a430 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:27.177026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eb6d3-1e82-4099-bb97-631ecca4a430 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:27.187800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391eb6d3-1e82-4099-bb97-631ecca4a430 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:27.192670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391eb6d3-1e82-4099-bb97-631ecca4a430 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:27.200460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 681/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:29.268974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a94529-f661-4ccd-8728-ec5bd39baae6 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:29.269486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a94529-f661-4ccd-8728-ec5bd39baae6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:29.269575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a94529-f661-4ccd-8728-ec5bd39baae6 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:29.270943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 699/1380] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:29.276472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40703a62-d8d7-4093-9727-28b4a119d883 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:29.276899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40703a62-d8d7-4093-9727-28b4a119d883 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:29.277058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40703a62-d8d7-4093-9727-28b4a119d883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:29.295932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-40703a62-d8d7-4093-9727-28b4a119d883 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:29.297165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-40703a62-d8d7-4093-9727-28b4a119d883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:29.305707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 682/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:29.350616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a94529-f661-4ccd-8728-ec5bd39baae6 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:29.367490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a94529-f661-4ccd-8728-ec5bd39baae6 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:29.374628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76a02d-3058-4ac2-8d92-67e5930c10d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:29.375388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76a02d-3058-4ac2-8d92-67e5930c10d1 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:29.375826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76a02d-3058-4ac2-8d92-67e5930c10d1 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:29.376204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76a02d-3058-4ac2-8d92-67e5930c10d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:29.384982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76a02d-3058-4ac2-8d92-67e5930c10d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:29.386624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76a02d-3058-4ac2-8d92-67e5930c10d1 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:29.396971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 700/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:31.506220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607b692-9907-44d9-a926-fb86bf9789db None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:31.506784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607b692-9907-44d9-a926-fb86bf9789db None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:31.506784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607b692-9907-44d9-a926-fb86bf9789db None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:31.508020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 683/1383] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:31.513702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd167-b429-4ae3-85a1-6e82d3ebc88e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:31.514276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd167-b429-4ae3-85a1-6e82d3ebc88e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:31.514276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd167-b429-4ae3-85a1-6e82d3ebc88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:31.533661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6c8fd167-b429-4ae3-85a1-6e82d3ebc88e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:31.534880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6c8fd167-b429-4ae3-85a1-6e82d3ebc88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:31.543998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 701/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:46:31.587436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607b692-9907-44d9-a926-fb86bf9789db None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:31.603094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607b692-9907-44d9-a926-fb86bf9789db None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:31.608812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca438b-3639-4cdd-9b56-1aa524120157 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:31.609467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca438b-3639-4cdd-9b56-1aa524120157 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:31.610028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca438b-3639-4cdd-9b56-1aa524120157 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:31.610312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca438b-3639-4cdd-9b56-1aa524120157 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:31.619706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca438b-3639-4cdd-9b56-1aa524120157 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:31.621379 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca438b-3639-4cdd-9b56-1aa524120157 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:31.632869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 684/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:33.703106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370e41e-864c-4d00-8de9-4e7a7fca6db6 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:33.703786 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370e41e-864c-4d00-8de9-4e7a7fca6db6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:33.703916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370e41e-864c-4d00-8de9-4e7a7fca6db6 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:33.705444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 702/1386] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:33.710857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c876ed9-1095-45ca-aa71-23295cca01d7 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:33.711301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c876ed9-1095-45ca-aa71-23295cca01d7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:33.711399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c876ed9-1095-45ca-aa71-23295cca01d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:33.730067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3c876ed9-1095-45ca-aa71-23295cca01d7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:33.731284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3c876ed9-1095-45ca-aa71-23295cca01d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:33.740164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 685/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:33.792954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370e41e-864c-4d00-8de9-4e7a7fca6db6 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:33.808251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370e41e-864c-4d00-8de9-4e7a7fca6db6 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:33.814117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b55838-92d5-43f5-b655-586c6372b45d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:33.814783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b55838-92d5-43f5-b655-586c6372b45d service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:33.815268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b55838-92d5-43f5-b655-586c6372b45d service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:33.815385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b55838-92d5-43f5-b655-586c6372b45d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:33.824341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b55838-92d5-43f5-b655-586c6372b45d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:33.826028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b55838-92d5-43f5-b655-586c6372b45d service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:33.836164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 703/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:46:35.963572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167c039-d4e8-4563-ad82-5d0007baea6e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:35.964320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167c039-d4e8-4563-ad82-5d0007baea6e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:35.964320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167c039-d4e8-4563-ad82-5d0007baea6e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:35.965768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 686/1389] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:46:35.971278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef05ad-193d-4942-b4cf-8a2c31625422 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:35.971915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef05ad-193d-4942-b4cf-8a2c31625422 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:35.971915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef05ad-193d-4942-b4cf-8a2c31625422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:35.991108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-60ef05ad-193d-4942-b4cf-8a2c31625422 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:35.992352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-60ef05ad-193d-4942-b4cf-8a2c31625422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:36.001492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 704/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:36.048861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167c039-d4e8-4563-ad82-5d0007baea6e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:36.064943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167c039-d4e8-4563-ad82-5d0007baea6e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:36.071688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9f483-644c-46c5-9fc6-befa41364c36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:36.072401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9f483-644c-46c5-9fc6-befa41364c36 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:36.072683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9f483-644c-46c5-9fc6-befa41364c36 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:36.073033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9f483-644c-46c5-9fc6-befa41364c36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:36.084750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9f483-644c-46c5-9fc6-befa41364c36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:36.086513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9f483-644c-46c5-9fc6-befa41364c36 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:36.097309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 687/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:38.290349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1b47-9715-41b8-bc84-dab63b64525e None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:38.291054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1b47-9715-41b8-bc84-dab63b64525e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:38.291054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1b47-9715-41b8-bc84-dab63b64525e None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:38.292584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 705/1392] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:38.298270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532f2d-50aa-4ce3-845c-4957ee6299ce None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:38.298716 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532f2d-50aa-4ce3-845c-4957ee6299ce None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:38.298847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67532f2d-50aa-4ce3-845c-4957ee6299ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:38.319567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-67532f2d-50aa-4ce3-845c-4957ee6299ce None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:38.321320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-67532f2d-50aa-4ce3-845c-4957ee6299ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:38.334278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 688/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:38.386302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb1b47-9715-41b8-bc84-dab63b64525e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:38.407463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb1b47-9715-41b8-bc84-dab63b64525e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:38.415335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfb22d-9e0f-4a97-b361-533687ae5a4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:38.416154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb22d-9e0f-4a97-b361-533687ae5a4f service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:38.416510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb22d-9e0f-4a97-b361-533687ae5a4f service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:38.416864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfb22d-9e0f-4a97-b361-533687ae5a4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:38.429244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfb22d-9e0f-4a97-b361-533687ae5a4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:38.431516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfb22d-9e0f-4a97-b361-533687ae5a4f service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:38.445701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 706/1394] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:40.557864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261697-14d5-4860-a9b2-c439b5171d8e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:40.558393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261697-14d5-4860-a9b2-c439b5171d8e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:40.558393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261697-14d5-4860-a9b2-c439b5171d8e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:40.559718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 689/1395] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:40.565374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af02a97-6633-4de0-b940-95d74b5eea0f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:40.565939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af02a97-6633-4de0-b940-95d74b5eea0f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:40.565939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af02a97-6633-4de0-b940-95d74b5eea0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:40.585944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3af02a97-6633-4de0-b940-95d74b5eea0f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:40.587179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3af02a97-6633-4de0-b940-95d74b5eea0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:40.595732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 707/1396] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:46:40.640477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261697-14d5-4860-a9b2-c439b5171d8e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:40.655976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261697-14d5-4860-a9b2-c439b5171d8e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:40.661913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62624c7b-73e4-4119-bb34-b0d7680ce330 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:40.662594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62624c7b-73e4-4119-bb34-b0d7680ce330 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:40.662865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62624c7b-73e4-4119-bb34-b0d7680ce330 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:40.663132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62624c7b-73e4-4119-bb34-b0d7680ce330 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:40.671988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62624c7b-73e4-4119-bb34-b0d7680ce330 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:40.673863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62624c7b-73e4-4119-bb34-b0d7680ce330 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:40.683753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 690/1397] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:41.385801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe705e1-72c7-41a2-ad4d-5951e36458fd None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:41.386061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe705e1-72c7-41a2-ad4d-5951e36458fd None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:41.386426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe705e1-72c7-41a2-ad4d-5951e36458fd None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:41.387761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 708/1398] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:41.393149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e137a7-eced-4124-b01f-077ccb63fc8b None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:41.393436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e137a7-eced-4124-b01f-077ccb63fc8b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:41.393717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e137a7-eced-4124-b01f-077ccb63fc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:41.412719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c2e137a7-eced-4124-b01f-077ccb63fc8b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:41.414037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c2e137a7-eced-4124-b01f-077ccb63fc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:41.423208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 691/1399] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:41.479431 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80ba46-75d2-4c9d-bd40-d491b9bc9fc5 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:41.479714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80ba46-75d2-4c9d-bd40-d491b9bc9fc5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:41.479988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80ba46-75d2-4c9d-bd40-d491b9bc9fc5 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:41.481439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 709/1400] 10.222.0.52 () {54 vars in 962 bytes} [Tue Jun 21 15:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:41.487173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4255032-e201-49c5-863e-dcfc9907d776 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:41.487460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4255032-e201-49c5-863e-dcfc9907d776 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:41.487734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4255032-e201-49c5-863e-dcfc9907d776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:41.541248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a4255032-e201-49c5-863e-dcfc9907d776 None None] MFA Rules not processed for user `5cb5dc4c5a5a448294c57a2db5938d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:41.542480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a4255032-e201-49c5-863e-dcfc9907d776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:41.577037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:46:41.609484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 692/1401] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Jun 21 15:46:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:41.617300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80ba46-75d2-4c9d-bd40-d491b9bc9fc5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:41.629566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80ba46-75d2-4c9d-bd40-d491b9bc9fc5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:41.633397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd24fd49-b002-4012-ac8c-129b9b8dba9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:41.634158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24fd49-b002-4012-ac8c-129b9b8dba9b service glance] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:41.634511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24fd49-b002-4012-ac8c-129b9b8dba9b service glance] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:41.634857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24fd49-b002-4012-ac8c-129b9b8dba9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:41.645798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24fd49-b002-4012-ac8c-129b9b8dba9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:41.647374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd24fd49-b002-4012-ac8c-129b9b8dba9b service glance] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:41.658202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 710/1402] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:42.396363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87357e2-797f-4f66-a1be-92acad33d936 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:42.396661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87357e2-797f-4f66-a1be-92acad33d936 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:42.396928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87357e2-797f-4f66-a1be-92acad33d936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:42.447757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a87357e2-797f-4f66-a1be-92acad33d936 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:42.449002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a87357e2-797f-4f66-a1be-92acad33d936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:42.517646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 693/1403] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:46:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:42.566808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df0fea-e18f-4412-aded-8ff14a1f3873 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:42.567117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df0fea-e18f-4412-aded-8ff14a1f3873 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:42.567398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df0fea-e18f-4412-aded-8ff14a1f3873 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:42.568838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 711/1404] 10.222.0.52 () {54 vars in 942 bytes} [Tue Jun 21 15:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:42.575327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ee15-48a4-4b63-ae70-156d647dc5da None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:42.575621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ee15-48a4-4b63-ae70-156d647dc5da None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:42.575898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ee15-48a4-4b63-ae70-156d647dc5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:42.630194 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0403ee15-48a4-4b63-ae70-156d647dc5da None None] MFA Rules not processed for user `7230dc79eb9844f1aa618d5e1c883b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:42.631416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0403ee15-48a4-4b63-ae70-156d647dc5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:42.712369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 694/1405] 10.222.0.52 () {58 vars in 1039 bytes} [Tue Jun 21 15:46:42 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:42.722357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df0fea-e18f-4412-aded-8ff14a1f3873 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:42.730119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=61 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:46:42.738271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df0fea-e18f-4412-aded-8ff14a1f3873 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:42.744560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4716f-4645-4e5d-b63d-38996407f0f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:42.744560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4716f-4645-4e5d-b63d-38996407f0f0 service swift] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:42.749415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4716f-4645-4e5d-b63d-38996407f0f0 service swift] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:42.749721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4716f-4645-4e5d-b63d-38996407f0f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:42.759579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4716f-4645-4e5d-b63d-38996407f0f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:42.761193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4716f-4645-4e5d-b63d-38996407f0f0 service swift] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:42.769718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 712/1406] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:46:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:46:42.785617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423eca3-400c-47bc-a3aa-b293daedbac7 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:42.785951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423eca3-400c-47bc-a3aa-b293daedbac7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:42.786325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423eca3-400c-47bc-a3aa-b293daedbac7 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:42.787762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 695/1407] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:46:42.793124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d19f1-6145-4aaf-8eb2-6915d89784c6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:42.793411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d19f1-6145-4aaf-8eb2-6915d89784c6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:42.793685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d19f1-6145-4aaf-8eb2-6915d89784c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:42.812615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-708d19f1-6145-4aaf-8eb2-6915d89784c6 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:46:42.814519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-708d19f1-6145-4aaf-8eb2-6915d89784c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:46:42.822553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 713/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:46:42.868066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423eca3-400c-47bc-a3aa-b293daedbac7 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:46:42.885332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423eca3-400c-47bc-a3aa-b293daedbac7 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:46:42.891008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b65099-d5d1-4009-9999-5c8ca3d9213e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:46:42.892114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b65099-d5d1-4009-9999-5c8ca3d9213e service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:46:42.892536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b65099-d5d1-4009-9999-5c8ca3d9213e service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:46:42.892918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b65099-d5d1-4009-9999-5c8ca3d9213e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:46:42.903120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b65099-d5d1-4009-9999-5c8ca3d9213e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:46:42.905057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b65099-d5d1-4009-9999-5c8ca3d9213e service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:46:42.911031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:46:42.916988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 696/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:46:52.807316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=8 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:47:05.081108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda555e-c567-4fbf-9f61-af068fa6aff5 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:05.081860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda555e-c567-4fbf-9f61-af068fa6aff5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:05.081860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda555e-c567-4fbf-9f61-af068fa6aff5 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:05.083068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 714/1410] 10.222.0.52 () {54 vars in 918 bytes} [Tue Jun 21 15:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:05.088784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c208b-f1d2-4d10-9b9e-c657b75fac9f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:05.089266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c208b-f1d2-4d10-9b9e-c657b75fac9f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:05.089381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c208b-f1d2-4d10-9b9e-c657b75fac9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:05.111051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-850c208b-f1d2-4d10-9b9e-c657b75fac9f None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:05.112269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-850c208b-f1d2-4d10-9b9e-c657b75fac9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:05.121965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 697/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Jun 21 15:47:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:05.137069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef609-0aca-40d8-bb14-a96f66c37f6d None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:05.137388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef609-0aca-40d8-bb14-a96f66c37f6d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:05.137672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef609-0aca-40d8-bb14-a96f66c37f6d None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:05.139056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 715/1412] 10.222.0.52 () {54 vars in 918 bytes} [Tue Jun 21 15:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:05.144845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb236a-deae-46be-a4d6-487847c73dfc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:05.145130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb236a-deae-46be-a4d6-487847c73dfc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:05.145439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb236a-deae-46be-a4d6-487847c73dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:05.164343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3dcb236a-deae-46be-a4d6-487847c73dfc None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:05.166757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3dcb236a-deae-46be-a4d6-487847c73dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:05.175857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 698/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Jun 21 15:47:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:07.773442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270ebd3-933b-4e6f-9f0f-4528346021d2 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:07.774019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270ebd3-933b-4e6f-9f0f-4528346021d2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:07.774019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270ebd3-933b-4e6f-9f0f-4528346021d2 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:07.775409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:07.780899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236578-0c74-4df4-915b-e08bab20fb5e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:07.781367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236578-0c74-4df4-915b-e08bab20fb5e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:07.781485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236578-0c74-4df4-915b-e08bab20fb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:07.806719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-47236578-0c74-4df4-915b-e08bab20fb5e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:07.808359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-47236578-0c74-4df4-915b-e08bab20fb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:07.817639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:07.861562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270ebd3-933b-4e6f-9f0f-4528346021d2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:07.878602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270ebd3-933b-4e6f-9f0f-4528346021d2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:07.884139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abc694-fe1c-434a-900e-944e624cd9cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:07.884780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abc694-fe1c-434a-900e-944e624cd9cb service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:07.885058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abc694-fe1c-434a-900e-944e624cd9cb service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:07.885331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abc694-fe1c-434a-900e-944e624cd9cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:07.896260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abc694-fe1c-434a-900e-944e624cd9cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:07.898173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40abc694-fe1c-434a-900e-944e624cd9cb service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:07.907657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 717/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:08.825420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e7dda-ab97-43ba-8bd4-2ac74628b796 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:08.826371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e7dda-ab97-43ba-8bd4-2ac74628b796 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:08.826371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e7dda-ab97-43ba-8bd4-2ac74628b796 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:08.828566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:47:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:08.833849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8991f-1154-4fcf-9257-0567fde3a1fb None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:08.834302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8991f-1154-4fcf-9257-0567fde3a1fb None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:08.834450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8991f-1154-4fcf-9257-0567fde3a1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:08.853357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-96e8991f-1154-4fcf-9257-0567fde3a1fb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:08.854600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-96e8991f-1154-4fcf-9257-0567fde3a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:08.862922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:09.021843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e7dda-ab97-43ba-8bd4-2ac74628b796 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:09.045963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e7dda-ab97-43ba-8bd4-2ac74628b796 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:09.052956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479e74d-29e5-460e-b16f-ec9566e8f5d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:09.053599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479e74d-29e5-460e-b16f-ec9566e8f5d2 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:09.053909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479e74d-29e5-460e-b16f-ec9566e8f5d2 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:09.054187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479e74d-29e5-460e-b16f-ec9566e8f5d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:09.064548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479e74d-29e5-460e-b16f-ec9566e8f5d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:09.066176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479e74d-29e5-460e-b16f-ec9566e8f5d2 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:09.076500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 701/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:09.983529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca619e8-1d6f-4c5d-84d3-cc2ea6d85e81 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:09.984125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca619e8-1d6f-4c5d-84d3-cc2ea6d85e81 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:09.984125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca619e8-1d6f-4c5d-84d3-cc2ea6d85e81 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:09.985545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:09.990927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97c9cf-48a8-48ef-a035-0739cd40fdc5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:09.991338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97c9cf-48a8-48ef-a035-0739cd40fdc5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:09.991489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97c9cf-48a8-48ef-a035-0739cd40fdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:10.012662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ab97c9cf-48a8-48ef-a035-0739cd40fdc5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:10.013938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ab97c9cf-48a8-48ef-a035-0739cd40fdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:10.022477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:10.173746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca619e8-1d6f-4c5d-84d3-cc2ea6d85e81 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:10.189788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca619e8-1d6f-4c5d-84d3-cc2ea6d85e81 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:10.195826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855be9a-0447-435e-9eb5-4ba2827f4427 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:10.196533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855be9a-0447-435e-9eb5-4ba2827f4427 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:10.196951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855be9a-0447-435e-9eb5-4ba2827f4427 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:10.197276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855be9a-0447-435e-9eb5-4ba2827f4427 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:10.206076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855be9a-0447-435e-9eb5-4ba2827f4427 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:10.207696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855be9a-0447-435e-9eb5-4ba2827f4427 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:10.217282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 720/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:11.085672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953abe15-0956-4bd4-903f-1af04379d041 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:11.086498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953abe15-0956-4bd4-903f-1af04379d041 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:11.086498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953abe15-0956-4bd4-903f-1af04379d041 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:11.087823 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:11.093815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00ac45-d8d9-45aa-87ae-d495c0441b3f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:11.094263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00ac45-d8d9-45aa-87ae-d495c0441b3f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:11.094371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00ac45-d8d9-45aa-87ae-d495c0441b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:11.116126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3f00ac45-d8d9-45aa-87ae-d495c0441b3f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:11.117335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3f00ac45-d8d9-45aa-87ae-d495c0441b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:11.126812 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:11.284467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953abe15-0956-4bd4-903f-1af04379d041 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:11.303274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953abe15-0956-4bd4-903f-1af04379d041 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:11.309348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf44d4a-9bae-4b2e-8d18-9eda4a2094d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:11.310055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf44d4a-9bae-4b2e-8d18-9eda4a2094d7 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:11.310341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf44d4a-9bae-4b2e-8d18-9eda4a2094d7 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:11.310618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf44d4a-9bae-4b2e-8d18-9eda4a2094d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:11.320113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf44d4a-9bae-4b2e-8d18-9eda4a2094d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:11.321797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf44d4a-9bae-4b2e-8d18-9eda4a2094d7 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:11.332335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 704/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:12.211736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6156b-b2f4-4a8d-b98a-b3babc2e2cb4 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:12.212310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6156b-b2f4-4a8d-b98a-b3babc2e2cb4 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:12.212310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6156b-b2f4-4a8d-b98a-b3babc2e2cb4 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:12.213763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:12.219094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923c8aa-8079-4dec-b9e2-83c182d6f6b5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:12.219537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923c8aa-8079-4dec-b9e2-83c182d6f6b5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:12.219657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923c8aa-8079-4dec-b9e2-83c182d6f6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:12.239752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1923c8aa-8079-4dec-b9e2-83c182d6f6b5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:12.241204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1923c8aa-8079-4dec-b9e2-83c182d6f6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:12.250452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:12.407334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6156b-b2f4-4a8d-b98a-b3babc2e2cb4 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:12.423387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6156b-b2f4-4a8d-b98a-b3babc2e2cb4 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:12.429548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1511b-b068-4ead-a6e4-37e9aaba6015 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:12.430242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1511b-b068-4ead-a6e4-37e9aaba6015 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:12.430528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1511b-b068-4ead-a6e4-37e9aaba6015 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:12.430799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1511b-b068-4ead-a6e4-37e9aaba6015 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:12.440103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1511b-b068-4ead-a6e4-37e9aaba6015 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:12.441926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1511b-b068-4ead-a6e4-37e9aaba6015 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:12.452697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 723/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:13.324187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d3c4-dbed-4d44-b2c3-875472a98f54 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:13.324731 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d3c4-dbed-4d44-b2c3-875472a98f54 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:13.324731 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d3c4-dbed-4d44-b2c3-875472a98f54 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:13.326234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:13.332220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8d2d2-4f18-4b6c-a8f5-3463acfc90b9 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:13.332790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8d2d2-4f18-4b6c-a8f5-3463acfc90b9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:13.332790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8d2d2-4f18-4b6c-a8f5-3463acfc90b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:13.352314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-7ae8d2d2-4f18-4b6c-a8f5-3463acfc90b9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:13.353514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7ae8d2d2-4f18-4b6c-a8f5-3463acfc90b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:13.363463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:13.515484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1d3c4-dbed-4d44-b2c3-875472a98f54 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:13.530315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1d3c4-dbed-4d44-b2c3-875472a98f54 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:13.535900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1299df-a65e-4f5a-bdaa-8fe29aa70735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:13.536553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1299df-a65e-4f5a-bdaa-8fe29aa70735 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:13.536833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1299df-a65e-4f5a-bdaa-8fe29aa70735 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:13.537108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1299df-a65e-4f5a-bdaa-8fe29aa70735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:13.546452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1299df-a65e-4f5a-bdaa-8fe29aa70735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:13.548087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1299df-a65e-4f5a-bdaa-8fe29aa70735 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:13.558839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 707/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:14.376470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c790021-5e04-4baa-a8b7-250c52f04e8e None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:14.377332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c790021-5e04-4baa-a8b7-250c52f04e8e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:14.377332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c790021-5e04-4baa-a8b7-250c52f04e8e None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:14.379544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:47:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:14.387671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271adcc-e09c-4baa-ab89-d4f8d350e17b None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:14.388387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271adcc-e09c-4baa-ab89-d4f8d350e17b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:14.388646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271adcc-e09c-4baa-ab89-d4f8d350e17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:14.410900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c271adcc-e09c-4baa-ab89-d4f8d350e17b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:14.412135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c271adcc-e09c-4baa-ab89-d4f8d350e17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:14.421653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:14.574172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c790021-5e04-4baa-a8b7-250c52f04e8e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:14.592048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c790021-5e04-4baa-a8b7-250c52f04e8e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:14.599574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15332a97-eb2a-43d0-a66d-cbfe10857cb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:14.600236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332a97-eb2a-43d0-a66d-cbfe10857cb4 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:14.600572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332a97-eb2a-43d0-a66d-cbfe10857cb4 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:14.600833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15332a97-eb2a-43d0-a66d-cbfe10857cb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:14.609901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15332a97-eb2a-43d0-a66d-cbfe10857cb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:14.612278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15332a97-eb2a-43d0-a66d-cbfe10857cb4 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:14.623066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 726/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:15.454303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d356-fb64-4d53-ac16-8ed1863b28db None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:15.454903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d356-fb64-4d53-ac16-8ed1863b28db None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:15.454903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72d356-fb64-4d53-ac16-8ed1863b28db None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:15.456406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:15.462431 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036a3e-db70-44a7-ba72-a1d9ea1ee894 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:15.462897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036a3e-db70-44a7-ba72-a1d9ea1ee894 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:15.463008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036a3e-db70-44a7-ba72-a1d9ea1ee894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:15.485718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2d036a3e-db70-44a7-ba72-a1d9ea1ee894 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:15.487159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2d036a3e-db70-44a7-ba72-a1d9ea1ee894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:15.500502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:15.666749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72d356-fb64-4d53-ac16-8ed1863b28db None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:15.681534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72d356-fb64-4d53-ac16-8ed1863b28db None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:15.687432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d324cc-9ab1-44bc-9cea-3c60acbe5a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:15.688117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d324cc-9ab1-44bc-9cea-3c60acbe5a86 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:15.688398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d324cc-9ab1-44bc-9cea-3c60acbe5a86 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:15.688669 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d324cc-9ab1-44bc-9cea-3c60acbe5a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:15.701062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d324cc-9ab1-44bc-9cea-3c60acbe5a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:15.703741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d324cc-9ab1-44bc-9cea-3c60acbe5a86 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:15.716987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 710/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:16.524334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551787d9-98de-477e-a295-a572e5411ec2 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:16.524871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551787d9-98de-477e-a295-a572e5411ec2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:16.524871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551787d9-98de-477e-a295-a572e5411ec2 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:16.526239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:16.531269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4d295-7e2f-4b2b-a755-866cbcb2f3c2 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:16.531705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4d295-7e2f-4b2b-a755-866cbcb2f3c2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:16.531817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4d295-7e2f-4b2b-a755-866cbcb2f3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:16.550535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1fc4d295-7e2f-4b2b-a755-866cbcb2f3c2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:16.551733 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1fc4d295-7e2f-4b2b-a755-866cbcb2f3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:16.559774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:16.710084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551787d9-98de-477e-a295-a572e5411ec2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:16.730643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551787d9-98de-477e-a295-a572e5411ec2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:16.737536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2c320-e7a1-4cde-836f-76ca138055a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:16.738285 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c320-e7a1-4cde-836f-76ca138055a8 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:16.738496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c320-e7a1-4cde-836f-76ca138055a8 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:16.738770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2c320-e7a1-4cde-836f-76ca138055a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:16.748564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2c320-e7a1-4cde-836f-76ca138055a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:16.750192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2c320-e7a1-4cde-836f-76ca138055a8 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:16.760959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 729/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:18.786745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86587bf-5969-4753-a1ad-40619e5ecc78 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:18.787362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86587bf-5969-4753-a1ad-40619e5ecc78 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:18.787362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86587bf-5969-4753-a1ad-40619e5ecc78 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:18.788745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:18.794536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ad9f4-5b1f-49c6-98eb-793a19857df7 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:18.794984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ad9f4-5b1f-49c6-98eb-793a19857df7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:18.795089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ad9f4-5b1f-49c6-98eb-793a19857df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:18.814811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1c7ad9f4-5b1f-49c6-98eb-793a19857df7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:18.816084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1c7ad9f4-5b1f-49c6-98eb-793a19857df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:18.825291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:18.867189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86587bf-5969-4753-a1ad-40619e5ecc78 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:18.868964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b86587bf-5969-4753-a1ad-40619e5ecc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:18.885790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86587bf-5969-4753-a1ad-40619e5ecc78 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:18.890527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dc455-bd73-4156-a859-60cdfdbf2159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:18.891207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dc455-bd73-4156-a859-60cdfdbf2159 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:18.891537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dc455-bd73-4156-a859-60cdfdbf2159 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:18.891898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dc455-bd73-4156-a859-60cdfdbf2159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:18.902821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dc455-bd73-4156-a859-60cdfdbf2159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:18.904285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dc455-bd73-4156-a859-60cdfdbf2159 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:18.915274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 713/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:21.293566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec76f1b-4fda-4ee0-a61d-ff108d542fd8 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:21.294158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec76f1b-4fda-4ee0-a61d-ff108d542fd8 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:21.294158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec76f1b-4fda-4ee0-a61d-ff108d542fd8 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:21.295716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:21.302136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3de71-4517-47c1-9a1a-c89cb35c6088 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:21.303275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3de71-4517-47c1-9a1a-c89cb35c6088 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:21.303275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3de71-4517-47c1-9a1a-c89cb35c6088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:21.333098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e3e3de71-4517-47c1-9a1a-c89cb35c6088 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:21.334327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e3e3de71-4517-47c1-9a1a-c89cb35c6088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:21.344440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:21.384365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec76f1b-4fda-4ee0-a61d-ff108d542fd8 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:21.396033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec76f1b-4fda-4ee0-a61d-ff108d542fd8 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:21.400246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780197e-748e-4049-ba9d-9c99b642f349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:21.400901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780197e-748e-4049-ba9d-9c99b642f349 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:21.401186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780197e-748e-4049-ba9d-9c99b642f349 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:21.401460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780197e-748e-4049-ba9d-9c99b642f349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:21.411930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780197e-748e-4049-ba9d-9c99b642f349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:21.413386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780197e-748e-4049-ba9d-9c99b642f349 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:21.423240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 732/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:23.821482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8af54-71cd-4217-9e20-ecf31f787a5b None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:23.822075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8af54-71cd-4217-9e20-ecf31f787a5b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:23.822075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8af54-71cd-4217-9e20-ecf31f787a5b None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:23.823335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:23.828832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a7b7f-bd25-4564-aa39-543a3f4a720d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:23.829319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a7b7f-bd25-4564-aa39-543a3f4a720d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:23.829439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a7b7f-bd25-4564-aa39-543a3f4a720d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:23.851498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1d4a7b7f-bd25-4564-aa39-543a3f4a720d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:23.852701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1d4a7b7f-bd25-4564-aa39-543a3f4a720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:23.862484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:47:23.904623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8af54-71cd-4217-9e20-ecf31f787a5b None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:23.915804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8af54-71cd-4217-9e20-ecf31f787a5b None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:23.919687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e88d7-e3c9-4c21-90bc-c930aa2b230d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:23.920335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e88d7-e3c9-4c21-90bc-c930aa2b230d service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:23.920618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e88d7-e3c9-4c21-90bc-c930aa2b230d service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:23.920896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e88d7-e3c9-4c21-90bc-c930aa2b230d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:23.930716 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e88d7-e3c9-4c21-90bc-c930aa2b230d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:23.932171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e88d7-e3c9-4c21-90bc-c930aa2b230d service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:23.943170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 716/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:26.177952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98134e-7a08-4923-9e8e-15a79a7264ee None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:26.178551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98134e-7a08-4923-9e8e-15a79a7264ee None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:26.178650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98134e-7a08-4923-9e8e-15a79a7264ee None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:26.180123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:26.186105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bbebc-59a5-4c69-ad7d-7a769f92b102 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:26.186775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bbebc-59a5-4c69-ad7d-7a769f92b102 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:26.186775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bbebc-59a5-4c69-ad7d-7a769f92b102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:26.214908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c60bbebc-59a5-4c69-ad7d-7a769f92b102 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:26.216215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c60bbebc-59a5-4c69-ad7d-7a769f92b102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:26.226605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:26.268166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98134e-7a08-4923-9e8e-15a79a7264ee None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:26.280718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98134e-7a08-4923-9e8e-15a79a7264ee None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:26.285802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18433ed7-0a7d-4dec-8930-16f21561c436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:26.286479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433ed7-0a7d-4dec-8930-16f21561c436 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:26.286762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433ed7-0a7d-4dec-8930-16f21561c436 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:26.287035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433ed7-0a7d-4dec-8930-16f21561c436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:26.299060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18433ed7-0a7d-4dec-8930-16f21561c436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:26.300860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18433ed7-0a7d-4dec-8930-16f21561c436 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:26.311889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 735/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:28.576180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1b4d6-fd00-4a63-98e3-e8f891fb816e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:28.576732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1b4d6-fd00-4a63-98e3-e8f891fb816e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:28.576732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1b4d6-fd00-4a63-98e3-e8f891fb816e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:28.577981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:28.583759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a7f47-cc66-46b5-ba3c-365f43743b84 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:28.584240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a7f47-cc66-46b5-ba3c-365f43743b84 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:28.584355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a7f47-cc66-46b5-ba3c-365f43743b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:28.602974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ee5a7f47-cc66-46b5-ba3c-365f43743b84 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:28.604191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ee5a7f47-cc66-46b5-ba3c-365f43743b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:28.612695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:47:28.658674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1b4d6-fd00-4a63-98e3-e8f891fb816e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:28.672028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1b4d6-fd00-4a63-98e3-e8f891fb816e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:28.676486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb30f9-5800-47ff-859d-b7c3754dac71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:28.677135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb30f9-5800-47ff-859d-b7c3754dac71 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:28.677417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb30f9-5800-47ff-859d-b7c3754dac71 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:28.677694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb30f9-5800-47ff-859d-b7c3754dac71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:28.687701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb30f9-5800-47ff-859d-b7c3754dac71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:28.689219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb30f9-5800-47ff-859d-b7c3754dac71 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:28.699291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 719/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:30.930165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763932e-858e-4543-b760-29ffe5722fba None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:30.930808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763932e-858e-4543-b760-29ffe5722fba None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:30.930808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763932e-858e-4543-b760-29ffe5722fba None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:30.932288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:30.938308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbad06-9ef9-49b6-a794-9cdd73ff0cba None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:30.938809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbad06-9ef9-49b6-a794-9cdd73ff0cba None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:30.939377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbad06-9ef9-49b6-a794-9cdd73ff0cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:30.960383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e0dbad06-9ef9-49b6-a794-9cdd73ff0cba None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:30.961655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e0dbad06-9ef9-49b6-a794-9cdd73ff0cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:30.970986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:31.013330 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763932e-858e-4543-b760-29ffe5722fba None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:31.026765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763932e-858e-4543-b760-29ffe5722fba None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:31.031732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dd73a-2b07-4fff-97ff-a578a297b6bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:31.032505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dd73a-2b07-4fff-97ff-a578a297b6bb service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:31.032764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dd73a-2b07-4fff-97ff-a578a297b6bb service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:31.033033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dd73a-2b07-4fff-97ff-a578a297b6bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:31.043478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9dd73a-2b07-4fff-97ff-a578a297b6bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:31.045046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9dd73a-2b07-4fff-97ff-a578a297b6bb service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:31.056198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 738/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:33.186671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62893fb0-b50a-416b-8011-09cbf7d7ce1a None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:33.187498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62893fb0-b50a-416b-8011-09cbf7d7ce1a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:33.187498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62893fb0-b50a-416b-8011-09cbf7d7ce1a None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:33.189501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:33.195503 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead375-ce56-401e-9b38-398ca691eeec None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:33.195972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead375-ce56-401e-9b38-398ca691eeec None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:33.196074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ead375-ce56-401e-9b38-398ca691eeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:33.204725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=70 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:47:33.218443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-38ead375-ce56-401e-9b38-398ca691eeec None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:33.219653 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-38ead375-ce56-401e-9b38-398ca691eeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:33.228108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:33.268159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62893fb0-b50a-416b-8011-09cbf7d7ce1a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:33.274868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=73 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:47:33.282038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62893fb0-b50a-416b-8011-09cbf7d7ce1a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:33.286138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9ba84-ec85-47ee-a099-06420417a90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:33.286808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9ba84-ec85-47ee-a099-06420417a90f service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:33.287100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9ba84-ec85-47ee-a099-06420417a90f service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:33.287379 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9ba84-ec85-47ee-a099-06420417a90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:33.297051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9ba84-ec85-47ee-a099-06420417a90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:33.298632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9ba84-ec85-47ee-a099-06420417a90f service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:33.308183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 722/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:35.463983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca5ba1-b0c7-477e-b89e-ac3da85d9e27 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:35.464562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca5ba1-b0c7-477e-b89e-ac3da85d9e27 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:35.464562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca5ba1-b0c7-477e-b89e-ac3da85d9e27 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:35.466557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:35.473796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc864b-1ac4-4a8c-a0f6-2c98794ed5f3 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:35.474316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc864b-1ac4-4a8c-a0f6-2c98794ed5f3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:35.474412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc864b-1ac4-4a8c-a0f6-2c98794ed5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:35.493738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-effc864b-1ac4-4a8c-a0f6-2c98794ed5f3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:35.495042 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-effc864b-1ac4-4a8c-a0f6-2c98794ed5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:35.505263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:47:35.543812 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca5ba1-b0c7-477e-b89e-ac3da85d9e27 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:35.555596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca5ba1-b0c7-477e-b89e-ac3da85d9e27 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:35.559437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a503857-5e9e-40bd-bd02-8143d1c5b9d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:35.560085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503857-5e9e-40bd-bd02-8143d1c5b9d0 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:35.560368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503857-5e9e-40bd-bd02-8143d1c5b9d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:35.560639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503857-5e9e-40bd-bd02-8143d1c5b9d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:35.572584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a503857-5e9e-40bd-bd02-8143d1c5b9d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:35.574120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a503857-5e9e-40bd-bd02-8143d1c5b9d0 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:35.585240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 741/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:37.691316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8709f-9d24-426f-8eeb-f316bccd3f76 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:37.691872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8709f-9d24-426f-8eeb-f316bccd3f76 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:37.691872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8709f-9d24-426f-8eeb-f316bccd3f76 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:37.693237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:37.699747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b947919-4319-48d8-8fe6-5f315f6f1129 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:37.700435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b947919-4319-48d8-8fe6-5f315f6f1129 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:37.700775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b947919-4319-48d8-8fe6-5f315f6f1129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:37.727974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2b947919-4319-48d8-8fe6-5f315f6f1129 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:37.729257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2b947919-4319-48d8-8fe6-5f315f6f1129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:37.738840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:47:37.780043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8709f-9d24-426f-8eeb-f316bccd3f76 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:37.793449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8709f-9d24-426f-8eeb-f316bccd3f76 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:37.799960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c9b4b-f2f2-4a8e-abeb-a90487c9ff7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:37.800889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c9b4b-f2f2-4a8e-abeb-a90487c9ff7b service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:37.801322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c9b4b-f2f2-4a8e-abeb-a90487c9ff7b service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:37.801741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c9b4b-f2f2-4a8e-abeb-a90487c9ff7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:37.813382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c9b4b-f2f2-4a8e-abeb-a90487c9ff7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:37.815192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c9b4b-f2f2-4a8e-abeb-a90487c9ff7b service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:37.825178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 725/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:39.913661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8002d-3703-4bc1-8357-a670d0ef2cdd None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:39.914600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8002d-3703-4bc1-8357-a670d0ef2cdd None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:39.914600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8002d-3703-4bc1-8357-a670d0ef2cdd None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:39.916771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:47:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:47:39.923408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdc849-5a20-4596-a8e0-4298c006b129 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:39.924351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdc849-5a20-4596-a8e0-4298c006b129 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:39.924638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdc849-5a20-4596-a8e0-4298c006b129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:39.943879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-13bdc849-5a20-4596-a8e0-4298c006b129 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:47:39.945094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-13bdc849-5a20-4596-a8e0-4298c006b129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:47:39.954284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:47:39.992782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8002d-3703-4bc1-8357-a670d0ef2cdd None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:47:40.008557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8002d-3703-4bc1-8357-a670d0ef2cdd None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:47:40.012995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e9db4-2632-470e-b794-1556b8f75be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:47:40.013672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e9db4-2632-470e-b794-1556b8f75be1 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:47:40.013994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e9db4-2632-470e-b794-1556b8f75be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:47:40.014271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e9db4-2632-470e-b794-1556b8f75be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:47:40.024536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e9db4-2632-470e-b794-1556b8f75be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:47:40.026019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e9db4-2632-470e-b794-1556b8f75be1 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:47:40.036131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 744/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:47:49.938452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=17 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:47:50.031285 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=17 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:48:00.075188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752826dd-4521-48da-861b-5ab691dfd477 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:00.075838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752826dd-4521-48da-861b-5ab691dfd477 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:00.075838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752826dd-4521-48da-861b-5ab691dfd477 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:00.077170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 727/1471] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:48:00.083130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583325ab-165d-42f7-b868-8cb5413ab979 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:00.083605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583325ab-165d-42f7-b868-8cb5413ab979 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:00.083711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583325ab-165d-42f7-b868-8cb5413ab979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:00.101793 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-583325ab-165d-42f7-b868-8cb5413ab979 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:48:00.103018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-583325ab-165d-42f7-b868-8cb5413ab979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:48:00.111504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 745/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:48:00.161262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c2a8c-8b1c-4f66-8edb-b74bd5cc1669 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:00.161565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c2a8c-8b1c-4f66-8edb-b74bd5cc1669 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:00.161949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c2a8c-8b1c-4f66-8edb-b74bd5cc1669 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:00.163366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 728/1473] 10.222.0.52 () {54 vars in 962 bytes} [Tue Jun 21 15:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:48:00.168314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96670ff8-730a-4be2-b149-beac52f97afc None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:00.168602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96670ff8-730a-4be2-b149-beac52f97afc None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:00.168886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96670ff8-730a-4be2-b149-beac52f97afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:00.188351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-96670ff8-730a-4be2-b149-beac52f97afc None None] MFA Rules not processed for user `5cb5dc4c5a5a448294c57a2db5938d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:48:00.189811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-96670ff8-730a-4be2-b149-beac52f97afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:48:00.196770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 746/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Jun 21 15:48:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:48:00.204158 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c2a8c-8b1c-4f66-8edb-b74bd5cc1669 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:48:00.215676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c2a8c-8b1c-4f66-8edb-b74bd5cc1669 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:48:00.219515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5242bbb-d96b-49e5-a380-dc95527fe5f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:48:00.220154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5242bbb-d96b-49e5-a380-dc95527fe5f5 service glance] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:00.220512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5242bbb-d96b-49e5-a380-dc95527fe5f5 service glance] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:00.220779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5242bbb-d96b-49e5-a380-dc95527fe5f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:00.230510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5242bbb-d96b-49e5-a380-dc95527fe5f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:48:00.231996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5242bbb-d96b-49e5-a380-dc95527fe5f5 service glance] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:48:00.243075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 729/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:48:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:48:02.333569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120683c-e61b-410b-b8d2-aa7fc651058e None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:02.334188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120683c-e61b-410b-b8d2-aa7fc651058e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:02.334188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120683c-e61b-410b-b8d2-aa7fc651058e None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:02.335534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 747/1476] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:48:02.341081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114b37-69ad-4561-90c8-a18532b069d8 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:02.341546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114b37-69ad-4561-90c8-a18532b069d8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:02.341652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114b37-69ad-4561-90c8-a18532b069d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:02.361696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-84114b37-69ad-4561-90c8-a18532b069d8 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:48:02.362996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-84114b37-69ad-4561-90c8-a18532b069d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:48:02.372389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 730/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:48:02.396505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120683c-e61b-410b-b8d2-aa7fc651058e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:48:02.411451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120683c-e61b-410b-b8d2-aa7fc651058e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:48:02.417027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc3e96-2b3c-48b1-87a5-319542a37ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:48:02.417704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3e96-2b3c-48b1-87a5-319542a37ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:02.418013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3e96-2b3c-48b1-87a5-319542a37ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:02.418283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc3e96-2b3c-48b1-87a5-319542a37ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:02.419028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc3e96-2b3c-48b1-87a5-319542a37ff7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:48:02.420863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING py.warnings [None req-92fc3e96-2b3c-48b1-87a5-319542a37ff7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:48:02.420863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: warnings.warn(msg) Jun 21 15:48:02.420863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: Jun 21 15:48:02.422125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc3e96-2b3c-48b1-87a5-319542a37ff7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:48:02.442845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 748/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Jun 21 15:48:02 2022] GET /identity/v3/projects => generated 2553 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:48:04.540032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4a42a-09a6-4ab3-8580-2ebdac87bf22 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:04.541054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4a42a-09a6-4ab3-8580-2ebdac87bf22 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:04.541054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4a42a-09a6-4ab3-8580-2ebdac87bf22 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:04.543248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 731/1479] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:48:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:48:04.549189 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bee1ea-c196-4e0d-98cf-2b6b2ceae0ff None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:04.549671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bee1ea-c196-4e0d-98cf-2b6b2ceae0ff None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:04.549796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bee1ea-c196-4e0d-98cf-2b6b2ceae0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:04.576645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-32bee1ea-c196-4e0d-98cf-2b6b2ceae0ff None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:48:04.578425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-32bee1ea-c196-4e0d-98cf-2b6b2ceae0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:48:04.597521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 749/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:48:04.645233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4a42a-09a6-4ab3-8580-2ebdac87bf22 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:48:04.663965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4a42a-09a6-4ab3-8580-2ebdac87bf22 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:48:04.670904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2203a5-3524-4520-b1ac-fad31d53a569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:48:04.671663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2203a5-3524-4520-b1ac-fad31d53a569 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:04.671985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2203a5-3524-4520-b1ac-fad31d53a569 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:04.672323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2203a5-3524-4520-b1ac-fad31d53a569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:04.681690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2203a5-3524-4520-b1ac-fad31d53a569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:48:04.683343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2203a5-3524-4520-b1ac-fad31d53a569 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:48:04.693325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 732/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:48:06.946082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddedeb6-f85d-4223-8eee-e7e7b06949ed None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:06.946665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddedeb6-f85d-4223-8eee-e7e7b06949ed None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:06.946665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddedeb6-f85d-4223-8eee-e7e7b06949ed None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:06.948820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 750/1482] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:48:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:48:06.956359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb04f4-0a03-4ad4-83ac-1fcf11d4181c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:06.956829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb04f4-0a03-4ad4-83ac-1fcf11d4181c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:06.956934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb04f4-0a03-4ad4-83ac-1fcf11d4181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:06.975915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-53cb04f4-0a03-4ad4-83ac-1fcf11d4181c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:48:06.977147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-53cb04f4-0a03-4ad4-83ac-1fcf11d4181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:48:06.985924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 733/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:48:07.027189 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddedeb6-f85d-4223-8eee-e7e7b06949ed None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:48:07.040396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddedeb6-f85d-4223-8eee-e7e7b06949ed None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:48:07.044690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e94ec-52af-4988-bfe3-acdd64215f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:48:07.045347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e94ec-52af-4988-bfe3-acdd64215f53 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:07.046387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e94ec-52af-4988-bfe3-acdd64215f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:07.046387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e94ec-52af-4988-bfe3-acdd64215f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:07.058078 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e94ec-52af-4988-bfe3-acdd64215f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:48:07.060052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e94ec-52af-4988-bfe3-acdd64215f53 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:48:07.070449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 751/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:48:09.200821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6a0f-d586-4bdf-8c50-6a6a30ed82be None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:09.201384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6a0f-d586-4bdf-8c50-6a6a30ed82be None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:09.201384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e6a0f-d586-4bdf-8c50-6a6a30ed82be None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:09.202830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 734/1485] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:48:09.210004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc4c59-66aa-4959-a3ca-b8e33fbbfb7a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:09.210493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc4c59-66aa-4959-a3ca-b8e33fbbfb7a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:09.211001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc4c59-66aa-4959-a3ca-b8e33fbbfb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:09.233984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6fdc4c59-66aa-4959-a3ca-b8e33fbbfb7a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:48:09.235223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6fdc4c59-66aa-4959-a3ca-b8e33fbbfb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:48:09.244921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 752/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:48:09.285706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e6a0f-d586-4bdf-8c50-6a6a30ed82be None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:48:09.298585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e6a0f-d586-4bdf-8c50-6a6a30ed82be None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:48:09.302541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adead92b-c0ac-48a3-882a-b93f3c145266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:48:09.303190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adead92b-c0ac-48a3-882a-b93f3c145266 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:09.303476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adead92b-c0ac-48a3-882a-b93f3c145266 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:09.303750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adead92b-c0ac-48a3-882a-b93f3c145266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:09.314083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adead92b-c0ac-48a3-882a-b93f3c145266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:48:09.315976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adead92b-c0ac-48a3-882a-b93f3c145266 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:48:09.326205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 735/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:48:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:48:11.552859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f9f3a-554d-4c30-837b-8fb56ffa6760 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:11.553706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f9f3a-554d-4c30-837b-8fb56ffa6760 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:11.553706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f9f3a-554d-4c30-837b-8fb56ffa6760 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:11.554718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 753/1488] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:48:11.561212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ddc6-2974-46d0-be18-bfbd8a750ccd None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:11.561663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ddc6-2974-46d0-be18-bfbd8a750ccd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:11.561768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038ddc6-2974-46d0-be18-bfbd8a750ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:11.581961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0038ddc6-2974-46d0-be18-bfbd8a750ccd None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:48:11.583214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0038ddc6-2974-46d0-be18-bfbd8a750ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:48:11.593364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 736/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:48:11.635656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f9f3a-554d-4c30-837b-8fb56ffa6760 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:48:11.647087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f9f3a-554d-4c30-837b-8fb56ffa6760 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:48:11.650989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2724f26-6e32-499e-a27a-1162a4c42a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:48:11.651651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724f26-6e32-499e-a27a-1162a4c42a96 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:11.651929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724f26-6e32-499e-a27a-1162a4c42a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:11.652206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724f26-6e32-499e-a27a-1162a4c42a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:11.663440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2724f26-6e32-499e-a27a-1162a4c42a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:48:11.665451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2724f26-6e32-499e-a27a-1162a4c42a96 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:48:11.675442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 754/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:48:13.925554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa80248-06a7-42b1-bc67-521fce1de1ca None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:13.926440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa80248-06a7-42b1-bc67-521fce1de1ca None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:13.926440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa80248-06a7-42b1-bc67-521fce1de1ca None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:13.928380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 737/1491] 10.222.0.52 () {54 vars in 926 bytes} [Tue Jun 21 15:48:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:48:13.937335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbba04c-1e08-4a9a-aa30-36c8ad9c1ac6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:13.937975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbba04c-1e08-4a9a-aa30-36c8ad9c1ac6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:13.938105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbba04c-1e08-4a9a-aa30-36c8ad9c1ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:13.961000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3fbba04c-1e08-4a9a-aa30-36c8ad9c1ac6 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:48:13.962337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3fbba04c-1e08-4a9a-aa30-36c8ad9c1ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:48:13.972224 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 755/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Tue Jun 21 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:48:14.011544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa80248-06a7-42b1-bc67-521fce1de1ca None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:48:14.025264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa80248-06a7-42b1-bc67-521fce1de1ca None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:48:14.031272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa59058-feb7-4146-872a-29db3adee96f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:48:14.031925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59058-feb7-4146-872a-29db3adee96f service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:48:14.032203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59058-feb7-4146-872a-29db3adee96f service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:48:14.032476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa59058-feb7-4146-872a-29db3adee96f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:48:14.043175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa59058-feb7-4146-872a-29db3adee96f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:48:14.044638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa59058-feb7-4146-872a-29db3adee96f service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:48:14.055413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 738/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:48:23.954902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=33 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:48:24.050338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=33 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:49:41.041686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b571-c664-4237-915d-874bbe22a31a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.042368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b571-c664-4237-915d-874bbe22a31a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.042368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259b571-c664-4237-915d-874bbe22a31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.061561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e259b571-c664-4237-915d-874bbe22a31a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:49:41.062800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e259b571-c664-4237-915d-874bbe22a31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:49:41.071091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 756/1494] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:49:41.080515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d28ac5-fa85-4b75-afbc-ed3e20bcc8dc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.081069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d28ac5-fa85-4b75-afbc-ed3e20bcc8dc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.081192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d28ac5-fa85-4b75-afbc-ed3e20bcc8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.106304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b5d28ac5-fa85-4b75-afbc-ed3e20bcc8dc None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:49:41.107500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b5d28ac5-fa85-4b75-afbc-ed3e20bcc8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:49:41.116015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 739/1495] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:49:41.120640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259b571-c664-4237-915d-874bbe22a31a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:41.131193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259b571-c664-4237-915d-874bbe22a31a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:41.135872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:41.136531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.136817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.137093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.137849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:49:41.138159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:41.139186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING py.warnings [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:49:41.139186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: warnings.warn(msg) Jun 21 15:49:41.139186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: Jun 21 15:49:41.140062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:41.153179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 757/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:49:41.162127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d28ac5-fa85-4b75-afbc-ed3e20bcc8dc None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:41.171774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d28ac5-fa85-4b75-afbc-ed3e20bcc8dc None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:41.177336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:41.178031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.178313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.178586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.179293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:41.181102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WARNING py.warnings [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:49:41.181102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: warnings.warn(msg) Jun 21 15:49:41.181102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: Jun 21 15:49:41.182397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:41.227494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 740/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:49:41 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:49:41.232512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:41.243205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106c44-031a-4c16-867b-d8afbc19611d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:41.300619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:41.301188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.301473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.301746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.302542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:41.303576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING py.warnings [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:49:41.303576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: warnings.warn(msg) Jun 21 15:49:41.303576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: Jun 21 15:49:41.304762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:41.325970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 758/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:49:41 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:49:41.332349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:41.344604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6aca70-621e-4ba7-8434-96936c4a60b8 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:41.350040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:41.350706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.350991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.351254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.351976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:41.353025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WARNING py.warnings [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:49:41.353025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: warnings.warn(msg) Jun 21 15:49:41.353025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: Jun 21 15:49:41.353949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:41.368715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 741/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:49:41.373531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:41.384033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb33d2-8037-4575-89b8-7ed3e2d89667 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:41.389021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645c635-8d55-4c38-a5bc-ee095bc1646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:41.389695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645c635-8d55-4c38-a5bc-ee095bc1646f admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.390007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645c635-8d55-4c38-a5bc-ee095bc1646f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.390289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645c635-8d55-4c38-a5bc-ee095bc1646f admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.391030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645c635-8d55-4c38-a5bc-ee095bc1646f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:41.392057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING py.warnings [None req-5645c635-8d55-4c38-a5bc-ee095bc1646f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:49:41.392057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: warnings.warn(msg) Jun 21 15:49:41.392057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: Jun 21 15:49:41.392979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645c635-8d55-4c38-a5bc-ee095bc1646f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:41.407001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 759/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:49:41.411958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:41.424549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b6ae0-4218-4fd0-9fcf-490c5be99005 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:41.431143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ca091-41c5-41a6-a07a-945efb5e644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:41.432498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ca091-41c5-41a6-a07a-945efb5e644f admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.432855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ca091-41c5-41a6-a07a-945efb5e644f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.433260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ca091-41c5-41a6-a07a-945efb5e644f admin admin] PATH_INFO: `/v3/projects/5291ebf4fdb74515b6f843b03b4f3cd9/users/16391fb5300744248d4dcdc188c2d7ab/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.452263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ca091-41c5-41a6-a07a-945efb5e644f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5291ebf4fdb74515b6f843b03b4f3cd9, user_id=16391fb5300744248d4dcdc188c2d7ab, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:41.453306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WARNING py.warnings [None req-171ca091-41c5-41a6-a07a-945efb5e644f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:49:41.453306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: warnings.warn(msg) Jun 21 15:49:41.453306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: Jun 21 15:49:41.455460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ca091-41c5-41a6-a07a-945efb5e644f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:41.473965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 742/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:49:41 2022] PUT /identity/v3/projects/5291ebf4fdb74515b6f843b03b4f3cd9/users/16391fb5300744248d4dcdc188c2d7ab/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:49:41.487901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc13305-3a76-42d5-91cb-063139f88cb2 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.488170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc13305-3a76-42d5-91cb-063139f88cb2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.488475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc13305-3a76-42d5-91cb-063139f88cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.528514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3cc13305-3a76-42d5-91cb-063139f88cb2 None None] MFA Rules not processed for user `16391fb5300744248d4dcdc188c2d7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:49:41.529758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3cc13305-3a76-42d5-91cb-063139f88cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:49:41.599677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 760/1502] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:49:41.615272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553388-0670-460f-8287-cc49f1ae6163 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.615609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553388-0670-460f-8287-cc49f1ae6163 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.615864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d553388-0670-460f-8287-cc49f1ae6163 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.617232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 743/1503] 10.222.0.52 () {54 vars in 963 bytes} [Tue Jun 21 15:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:49:41.622546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61f9c8-8c77-4720-bafd-b2b8f1df9077 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.622857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61f9c8-8c77-4720-bafd-b2b8f1df9077 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.623139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61f9c8-8c77-4720-bafd-b2b8f1df9077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.644406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5b61f9c8-8c77-4720-bafd-b2b8f1df9077 None None] MFA Rules not processed for user `ed0934c0b1a64dfca3431dc32441e2db`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:49:41.645658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5b61f9c8-8c77-4720-bafd-b2b8f1df9077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:49:41.696109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:49:41.706275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 761/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Tue Jun 21 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:49:41.714859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d553388-0670-460f-8287-cc49f1ae6163 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:41.731184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d553388-0670-460f-8287-cc49f1ae6163 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:41.735521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091a537-17da-4922-9e66-09944e0809cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:41.736624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a537-17da-4922-9e66-09944e0809cf service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.737030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a537-17da-4922-9e66-09944e0809cf service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.737393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a537-17da-4922-9e66-09944e0809cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.747564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091a537-17da-4922-9e66-09944e0809cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:41.749004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091a537-17da-4922-9e66-09944e0809cf service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:41.759142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 744/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:49:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:49:41.805453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61147bb1-2353-49a5-a0bd-bb1d5190ced7 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.805741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61147bb1-2353-49a5-a0bd-bb1d5190ced7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.806051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61147bb1-2353-49a5-a0bd-bb1d5190ced7 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.807417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 762/1506] 10.222.0.52 () {52 vars in 829 bytes} [Tue Jun 21 15:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:49:41.811383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091a537-17da-4922-9e66-09944e0809cf service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:41.895360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091a537-17da-4922-9e66-09944e0809cf service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:41.900325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:41.901342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:41.903365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:41.903659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] PATH_INFO: `/v3/users/16391fb5300744248d4dcdc188c2d7ab` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:41.905536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16391fb5300744248d4dcdc188c2d7ab)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:41.906614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WARNING py.warnings [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:49:41.906614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: warnings.warn(msg) Jun 21 15:49:41.906614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: Jun 21 15:49:41.907926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:41.990940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16391fb5300744248d4dcdc188c2d7ab'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.011857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16391fb5300744248d4dcdc188c2d7ab'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.022982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16391fb5300744248d4dcdc188c2d7ab'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.032847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:49:42.038780 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16391fb5300744248d4dcdc188c2d7ab'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.039961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16391fb5300744248d4dcdc188c2d7ab'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.047088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 745/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:49:41 2022] DELETE /identity/v3/users/16391fb5300744248d4dcdc188c2d7ab => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:49:42.054190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f96abe-8de2-4716-b009-87137d451605 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:42.054762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f96abe-8de2-4716-b009-87137d451605 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:42.054762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f96abe-8de2-4716-b009-87137d451605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:42.074378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e2f96abe-8de2-4716-b009-87137d451605 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:49:42.075664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e2f96abe-8de2-4716-b009-87137d451605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:49:42.142404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 763/1508] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:49:42.153621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:42.155240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:49:42.218724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbaa79-fa91-43c3-a0ec-c0bbae97e954 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:42.224297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20cf3b-d784-454f-a49f-aee6ffed17e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:42.225031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20cf3b-d784-454f-a49f-aee6ffed17e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:42.225331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20cf3b-d784-454f-a49f-aee6ffed17e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:42.225600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20cf3b-d784-454f-a49f-aee6ffed17e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:42.240453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20cf3b-d784-454f-a49f-aee6ffed17e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:42.241978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20cf3b-d784-454f-a49f-aee6ffed17e6 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:42.253015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 746/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:49:42.639399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f96abe-8de2-4716-b009-87137d451605 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:49:42.641438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e2f96abe-8de2-4716-b009-87137d451605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:49:42.655366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f96abe-8de2-4716-b009-87137d451605 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:49:42.661341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:49:42.662248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:49:42.662526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:49:42.662796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] PATH_INFO: `/v3/projects/5291ebf4fdb74515b6f843b03b4f3cd9` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:49:42.664595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5291ebf4fdb74515b6f843b03b4f3cd9)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:49:42.665637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING py.warnings [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:49:42.665637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: warnings.warn(msg) Jun 21 15:49:42.665637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: Jun 21 15:49:42.666862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:49:42.733268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5291ebf4fdb74515b6f843b03b4f3cd9', 'request_id': 'req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.745796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5291ebf4fdb74515b6f843b03b4f3cd9', 'request_id': 'req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.762710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5291ebf4fdb74515b6f843b03b4f3cd9', 'request_id': 'req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.775356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] The token cache is being invalidate because project 5291ebf4fdb74515b6f843b03b4f3cd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:49:42.775642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.776493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f1bdd292-46c0-4fb3-83d6-cfe4984c3f26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:49:42.778187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 764/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:49:42 2022] DELETE /identity/v3/projects/5291ebf4fdb74515b6f843b03b4f3cd9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:49:52.248338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=29 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:49:52.771898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:51:26.142711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461baa4-e07e-4a01-9eeb-210f6b05b0a0 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:26.143349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461baa4-e07e-4a01-9eeb-210f6b05b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:26.143349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461baa4-e07e-4a01-9eeb-210f6b05b0a0 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:26.144646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:26.150168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318cffc-6e0a-42af-ab69-7e811cec8f1a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:26.150645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318cffc-6e0a-42af-ab69-7e811cec8f1a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:26.150746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318cffc-6e0a-42af-ab69-7e811cec8f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:26.171257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1318cffc-6e0a-42af-ab69-7e811cec8f1a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:26.172466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1318cffc-6e0a-42af-ab69-7e811cec8f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:26.235554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:26.397639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461baa4-e07e-4a01-9eeb-210f6b05b0a0 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:26.400250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b461baa4-e07e-4a01-9eeb-210f6b05b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:26.485401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461baa4-e07e-4a01-9eeb-210f6b05b0a0 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:26.491575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df507d34-3d5f-4f92-8309-e70ac137a1c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:26.492253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507d34-3d5f-4f92-8309-e70ac137a1c8 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:26.492552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507d34-3d5f-4f92-8309-e70ac137a1c8 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:26.493184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507d34-3d5f-4f92-8309-e70ac137a1c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:26.502515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df507d34-3d5f-4f92-8309-e70ac137a1c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:26.504159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df507d34-3d5f-4f92-8309-e70ac137a1c8 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:26.514578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 748/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:51:27.389610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197edc7-fee6-4a7a-8f38-2399a9c1706a None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:27.390240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197edc7-fee6-4a7a-8f38-2399a9c1706a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:27.390240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197edc7-fee6-4a7a-8f38-2399a9c1706a None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:27.391620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:27.397526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f65aa-0942-496e-8e1b-2e5fdc91c0a9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:27.398080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f65aa-0942-496e-8e1b-2e5fdc91c0a9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:27.398274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f65aa-0942-496e-8e1b-2e5fdc91c0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:27.422740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d53f65aa-0942-496e-8e1b-2e5fdc91c0a9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:27.423988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d53f65aa-0942-496e-8e1b-2e5fdc91c0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:27.432678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:27.585242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197edc7-fee6-4a7a-8f38-2399a9c1706a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:27.600804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197edc7-fee6-4a7a-8f38-2399a9c1706a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:27.606565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebb7d9-3848-4022-9942-bc9d0bb775b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:27.607208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebb7d9-3848-4022-9942-bc9d0bb775b0 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:27.607491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebb7d9-3848-4022-9942-bc9d0bb775b0 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:27.607760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebb7d9-3848-4022-9942-bc9d0bb775b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:27.616508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebb7d9-3848-4022-9942-bc9d0bb775b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:27.618489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebb7d9-3848-4022-9942-bc9d0bb775b0 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:27.628411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 767/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:51:28.789483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7c39d-ba32-49d4-a738-88c91b19c445 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:28.790165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7c39d-ba32-49d4-a738-88c91b19c445 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:28.790165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7c39d-ba32-49d4-a738-88c91b19c445 None None] PATH_INFO: `/v3` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:28.791484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 750/1517] 10.222.0.23 () {54 vars in 925 bytes} [Tue Jun 21 15:51:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:51:28.798237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd565-067b-4eb3-a5f6-2b1b65e7142d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:28.798684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd565-067b-4eb3-a5f6-2b1b65e7142d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:28.798815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd565-067b-4eb3-a5f6-2b1b65e7142d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:28.818932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-7cddd565-067b-4eb3-a5f6-2b1b65e7142d None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:28.820134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7cddd565-067b-4eb3-a5f6-2b1b65e7142d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:28.887815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 768/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Jun 21 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:32.730812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecee792-10cb-44c8-aaaa-248b3ed46e2e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:32.731381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecee792-10cb-44c8-aaaa-248b3ed46e2e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:32.731381 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecee792-10cb-44c8-aaaa-248b3ed46e2e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:32.732733 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 751/1519] 10.222.0.52 () {54 vars in 960 bytes} [Tue Jun 21 15:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:32.737997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ee075-6536-4a00-a71f-502e28762f7d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:32.738518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ee075-6536-4a00-a71f-502e28762f7d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:32.738652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ee075-6536-4a00-a71f-502e28762f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:32.759677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-448ee075-6536-4a00-a71f-502e28762f7d None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:32.761090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-448ee075-6536-4a00-a71f-502e28762f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:32.775443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 769/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Tue Jun 21 15:51:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:32.783451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecee792-10cb-44c8-aaaa-248b3ed46e2e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:32.800297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecee792-10cb-44c8-aaaa-248b3ed46e2e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:32.807008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5a24c-fd86-40d4-bb44-8fae899e4f5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:32.807825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5a24c-fd86-40d4-bb44-8fae899e4f5b service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:32.808116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5a24c-fd86-40d4-bb44-8fae899e4f5b service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:32.808398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5a24c-fd86-40d4-bb44-8fae899e4f5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:32.827692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5a24c-fd86-40d4-bb44-8fae899e4f5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:32.829678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5a24c-fd86-40d4-bb44-8fae899e4f5b service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:32.840911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 752/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:51:38.890407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6aac-adbc-4ff5-a306-680d53ca118f None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:38.890993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6aac-adbc-4ff5-a306-680d53ca118f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:38.890993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6aac-adbc-4ff5-a306-680d53ca118f None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:38.892451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:38.897895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b5266-244b-478b-83a1-fd446f39f808 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:38.898362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b5266-244b-478b-83a1-fd446f39f808 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:38.898462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b5266-244b-478b-83a1-fd446f39f808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:38.920593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-928b5266-244b-478b-83a1-fd446f39f808 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:38.922569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-928b5266-244b-478b-83a1-fd446f39f808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:38.932044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:39.086471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf6aac-adbc-4ff5-a306-680d53ca118f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:39.102183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf6aac-adbc-4ff5-a306-680d53ca118f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:39.108955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f657fca-4c83-4714-a6a6-c63dfb0f3a59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:39.109760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f657fca-4c83-4714-a6a6-c63dfb0f3a59 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:39.110084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f657fca-4c83-4714-a6a6-c63dfb0f3a59 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:39.110361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f657fca-4c83-4714-a6a6-c63dfb0f3a59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:39.122378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f657fca-4c83-4714-a6a6-c63dfb0f3a59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:39.124300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f657fca-4c83-4714-a6a6-c63dfb0f3a59 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:39.135618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 771/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:51:39.958585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e025c-f4b6-42dc-81fb-bcd33ed793e8 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:39.959124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e025c-f4b6-42dc-81fb-bcd33ed793e8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:39.959124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e025c-f4b6-42dc-81fb-bcd33ed793e8 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:39.960530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 21 15:51:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:39.965865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488f43-f740-46de-b875-b997dee14098 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:39.966283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488f43-f740-46de-b875-b997dee14098 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:39.966490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488f43-f740-46de-b875-b997dee14098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:39.986438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-70488f43-f740-46de-b875-b997dee14098 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:39.987655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-70488f43-f740-46de-b875-b997dee14098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:39.998045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 21 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:40.154591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e025c-f4b6-42dc-81fb-bcd33ed793e8 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:40.170781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e025c-f4b6-42dc-81fb-bcd33ed793e8 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:40.176912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6e79a-0bc2-4acd-937a-8b5efe89e17b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:40.177582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e79a-0bc2-4acd-937a-8b5efe89e17b service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:40.177876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e79a-0bc2-4acd-937a-8b5efe89e17b service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:40.178159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e79a-0bc2-4acd-937a-8b5efe89e17b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:40.187245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6e79a-0bc2-4acd-937a-8b5efe89e17b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:40.188936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6e79a-0bc2-4acd-937a-8b5efe89e17b service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:40.198845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 755/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:51:44.551416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3f2f3-9927-4cf2-9a3d-be4df0daf0c2 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:44.551974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3f2f3-9927-4cf2-9a3d-be4df0daf0c2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:44.551974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3f2f3-9927-4cf2-9a3d-be4df0daf0c2 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:44.554026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:51:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:44.559633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad839-b186-4707-96c8-762171cd4f0e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:44.560045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad839-b186-4707-96c8-762171cd4f0e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:44.560202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ad839-b186-4707-96c8-762171cd4f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:44.580460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-817ad839-b186-4707-96c8-762171cd4f0e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:44.581751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-817ad839-b186-4707-96c8-762171cd4f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:44.591422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:44.631919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3f2f3-9927-4cf2-9a3d-be4df0daf0c2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:44.633802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b5e3f2f3-9927-4cf2-9a3d-be4df0daf0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:44.701394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3f2f3-9927-4cf2-9a3d-be4df0daf0c2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:44.705718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849aad97-3f1c-4334-8cbf-ce0cff51f156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:44.706760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aad97-3f1c-4334-8cbf-ce0cff51f156 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:44.707304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aad97-3f1c-4334-8cbf-ce0cff51f156 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:44.708036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aad97-3f1c-4334-8cbf-ce0cff51f156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:44.718432 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aad97-3f1c-4334-8cbf-ce0cff51f156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:44.720326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aad97-3f1c-4334-8cbf-ce0cff51f156 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:44.726710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:51:44.732621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 774/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:51:46.863334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061da0f2-7ce0-4446-84e6-38c4191919cd None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:46.864227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061da0f2-7ce0-4446-84e6-38c4191919cd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:46.864515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061da0f2-7ce0-4446-84e6-38c4191919cd None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:46.865922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:46.871858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9162ec1-6950-4946-8b7b-7c464cf768e6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:46.872346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9162ec1-6950-4946-8b7b-7c464cf768e6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:46.872437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9162ec1-6950-4946-8b7b-7c464cf768e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:46.893996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b9162ec1-6950-4946-8b7b-7c464cf768e6 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:46.895999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b9162ec1-6950-4946-8b7b-7c464cf768e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:46.904495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:46.946085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061da0f2-7ce0-4446-84e6-38c4191919cd None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:46.948058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-061da0f2-7ce0-4446-84e6-38c4191919cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:46.963588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061da0f2-7ce0-4446-84e6-38c4191919cd None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:46.968859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb83f66-c277-4347-a115-1f60d1fffafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:46.969519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb83f66-c277-4347-a115-1f60d1fffafd service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:46.969806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb83f66-c277-4347-a115-1f60d1fffafd service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:46.970182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb83f66-c277-4347-a115-1f60d1fffafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:46.980394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb83f66-c277-4347-a115-1f60d1fffafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:46.981917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb83f66-c277-4347-a115-1f60d1fffafd service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:46.992833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 758/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:51:50.801755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f0475-b382-4c94-be78-6188d3d1dcb7 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:50.802507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f0475-b382-4c94-be78-6188d3d1dcb7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:50.802507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f0475-b382-4c94-be78-6188d3d1dcb7 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:50.803992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:50.809517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c35db2-f89c-46d9-a84c-d08f650f0337 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:50.810021 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c35db2-f89c-46d9-a84c-d08f650f0337 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:50.810126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c35db2-f89c-46d9-a84c-d08f650f0337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:50.831544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-63c35db2-f89c-46d9-a84c-d08f650f0337 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:50.832747 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-63c35db2-f89c-46d9-a84c-d08f650f0337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:50.842249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:51.144887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f0475-b382-4c94-be78-6188d3d1dcb7 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:51.146752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-804f0475-b382-4c94-be78-6188d3d1dcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:51.229478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f0475-b382-4c94-be78-6188d3d1dcb7 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:51.233302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06812c1-69be-4f4d-8f65-a11f5fdb7320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:51.234032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06812c1-69be-4f4d-8f65-a11f5fdb7320 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:51.234311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06812c1-69be-4f4d-8f65-a11f5fdb7320 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:51.234590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06812c1-69be-4f4d-8f65-a11f5fdb7320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:51.249477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06812c1-69be-4f4d-8f65-a11f5fdb7320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:51.251222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06812c1-69be-4f4d-8f65-a11f5fdb7320 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:51.262699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 777/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:51:53.350096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5d672-b8a6-40f4-8327-d2823140fd3d None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:53.350690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5d672-b8a6-40f4-8327-d2823140fd3d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:53.350690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5d672-b8a6-40f4-8327-d2823140fd3d None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:53.352365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 21 15:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:51:53.357940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734621b-aefb-4b0f-848f-83af1a785d66 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:53.358403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734621b-aefb-4b0f-848f-83af1a785d66 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:53.358498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734621b-aefb-4b0f-848f-83af1a785d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:53.379046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d734621b-aefb-4b0f-848f-83af1a785d66 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:51:53.380695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d734621b-aefb-4b0f-848f-83af1a785d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:53.390034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 21 15:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:51:53.711101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5d672-b8a6-40f4-8327-d2823140fd3d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:51:53.712866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-67f5d672-b8a6-40f4-8327-d2823140fd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:51:53.720369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=65 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:51:53.726975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5d672-b8a6-40f4-8327-d2823140fd3d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:51:53.730551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85bd83-2b61-42f2-b1f6-7c00d1be6fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:51:53.731192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85bd83-2b61-42f2-b1f6-7c00d1be6fbc service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:51:53.731470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85bd83-2b61-42f2-b1f6-7c00d1be6fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:51:53.731740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85bd83-2b61-42f2-b1f6-7c00d1be6fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:51:53.741633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85bd83-2b61-42f2-b1f6-7c00d1be6fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:51:53.743119 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85bd83-2b61-42f2-b1f6-7c00d1be6fbc service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:51:53.753557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 761/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:52:03.372676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=32 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:52:03.748629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:23.346337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a5796-fd4e-4152-afe9-e9409ee4d3df None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.346995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a5796-fd4e-4152-afe9-e9409ee4d3df None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.346995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a5796-fd4e-4152-afe9-e9409ee4d3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.372977 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-cb7a5796-fd4e-4152-afe9-e9409ee4d3df None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:23.374201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cb7a5796-fd4e-4152-afe9-e9409ee4d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:23.386532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 779/1540] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:23.395696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2232b-8801-4e48-a82b-e98518f59240 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.397239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2232b-8801-4e48-a82b-e98518f59240 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.397851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def2232b-8801-4e48-a82b-e98518f59240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.427586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-def2232b-8801-4e48-a82b-e98518f59240 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:23.429858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-def2232b-8801-4e48-a82b-e98518f59240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:23.445659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 762/1541] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:23.450805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a5796-fd4e-4152-afe9-e9409ee4d3df None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:23.457829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480da5b-104d-4cd0-8695-69ca5e086b34 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.458501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480da5b-104d-4cd0-8695-69ca5e086b34 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.459002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480da5b-104d-4cd0-8695-69ca5e086b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.465820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a5796-fd4e-4152-afe9-e9409ee4d3df None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:23.485383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:23.486838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.487357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.487862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.488842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:23.496510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:23.505615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:23.509375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1480da5b-104d-4cd0-8695-69ca5e086b34 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:23.515715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1480da5b-104d-4cd0-8695-69ca5e086b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:23.541257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 780/1542] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:23.541976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 763/1543] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:53:23.550607 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:23.553354 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43babb-e1f8-45d3-94df-6a1e075eb512 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.555042 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43babb-e1f8-45d3-94df-6a1e075eb512 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.559506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43babb-e1f8-45d3-94df-6a1e075eb512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.602689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb33a64-7243-4591-862a-dfad6d68828e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:23.629630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-bc43babb-e1f8-45d3-94df-6a1e075eb512 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:23.632176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:23.633192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.633707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.635457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bc43babb-e1f8-45d3-94df-6a1e075eb512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:23.636039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.636982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:23.638978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING py.warnings [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:53:23.638978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: warnings.warn(msg) Jun 21 15:53:23.638978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: Jun 21 15:53:23.643474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:23.669045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 764/1544] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:23.675515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43babb-e1f8-45d3-94df-6a1e075eb512 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:23.689276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 781/1545] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:23.693401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:23.695263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43babb-e1f8-45d3-94df-6a1e075eb512 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:23.825133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945f3cb-8b54-44d9-ad3f-9b398bcc5f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:23.825900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f3cb-8b54-44d9-ad3f-9b398bcc5f6a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.826287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f3cb-8b54-44d9-ad3f-9b398bcc5f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.827339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945f3cb-8b54-44d9-ad3f-9b398bcc5f6a admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.828575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945f3cb-8b54-44d9-ad3f-9b398bcc5f6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:23.828909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945f3cb-8b54-44d9-ad3f-9b398bcc5f6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:23.830002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WARNING py.warnings [None req-4945f3cb-8b54-44d9-ad3f-9b398bcc5f6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:53:23.830002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: warnings.warn(msg) Jun 21 15:53:23.830002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: Jun 21 15:53:23.830938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945f3cb-8b54-44d9-ad3f-9b398bcc5f6a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:23.857809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 765/1546] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:53:23.859190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044759b6-3bd8-4582-88e6-9aab13dbda4e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.859484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044759b6-3bd8-4582-88e6-9aab13dbda4e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.859872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044759b6-3bd8-4582-88e6-9aab13dbda4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.893152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c28727-d1b7-4d9f-9033-83e1d1f4e716 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:23.907827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34e752-6fb5-4dfd-9caf-4a92ebdefcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:23.908825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34e752-6fb5-4dfd-9caf-4a92ebdefcf2 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.909235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34e752-6fb5-4dfd-9caf-4a92ebdefcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.911843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-044759b6-3bd8-4582-88e6-9aab13dbda4e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:23.912451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34e752-6fb5-4dfd-9caf-4a92ebdefcf2 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.914829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-044759b6-3bd8-4582-88e6-9aab13dbda4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:23.916499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34e752-6fb5-4dfd-9caf-4a92ebdefcf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:23.921791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34e752-6fb5-4dfd-9caf-4a92ebdefcf2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:23.935843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 766/1547] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:23.939396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68a9b-7d58-4f84-86cc-8e7d316a9833 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.940218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68a9b-7d58-4f84-86cc-8e7d316a9833 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.940718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68a9b-7d58-4f84-86cc-8e7d316a9833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.962640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 782/1548] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/users => generated 282 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:23.964588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd18c9-8a0a-474a-adbb-a688067df08d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.965008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd18c9-8a0a-474a-adbb-a688067df08d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.965298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd18c9-8a0a-474a-adbb-a688067df08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.968322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-08a68a9b-7d58-4f84-86cc-8e7d316a9833 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:23.969998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-08a68a9b-7d58-4f84-86cc-8e7d316a9833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:23.980090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 767/1549] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:23.981829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a68a9b-7d58-4f84-86cc-8e7d316a9833 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:23.987591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f0fd18c9-8a0a-474a-adbb-a688067df08d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:23.989046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f0fd18c9-8a0a-474a-adbb-a688067df08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:23.993398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a68a9b-7d58-4f84-86cc-8e7d316a9833 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:23.998098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3c089-3f68-424a-8f58-64bb9cc1521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:23.998453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 783/1550] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:23.998695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3c089-3f68-424a-8f58-64bb9cc1521f admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:23.999109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3c089-3f68-424a-8f58-64bb9cc1521f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:23.999391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3c089-3f68-424a-8f58-64bb9cc1521f admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:23.999954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c3cea-1839-4820-9666-3bae145204b2 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.000136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3c089-3f68-424a-8f58-64bb9cc1521f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.000345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c3cea-1839-4820-9666-3bae145204b2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.000626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c3cea-1839-4820-9666-3bae145204b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.002091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3c089-3f68-424a-8f58-64bb9cc1521f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.019903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5f0c3cea-1839-4820-9666-3bae145204b2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.021120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5f0c3cea-1839-4820-9666-3bae145204b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.034112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 784/1551] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.035225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c0d6-8faa-46e3-834f-e5465744050a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.035514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c0d6-8faa-46e3-834f-e5465744050a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.035790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c0d6-8faa-46e3-834f-e5465744050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.043951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 768/1552] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:23 2022] POST /identity/v3/projects => generated 405 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.045301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bce61-a881-4fb5-9cf7-eeda7b566ff2 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.045593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bce61-a881-4fb5-9cf7-eeda7b566ff2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.045941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bce61-a881-4fb5-9cf7-eeda7b566ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.057547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-7bf3c0d6-8faa-46e3-834f-e5465744050a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.058804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7bf3c0d6-8faa-46e3-834f-e5465744050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.064557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1d2bce61-a881-4fb5-9cf7-eeda7b566ff2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.065839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1d2bce61-a881-4fb5-9cf7-eeda7b566ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.121278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 785/1553] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.122353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3c0d6-8faa-46e3-834f-e5465744050a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.129619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 769/1554] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.131122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265bdd5-7081-4294-ad73-c30024010710 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.131415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265bdd5-7081-4294-ad73-c30024010710 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.131872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265bdd5-7081-4294-ad73-c30024010710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.134129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3c0d6-8faa-46e3-834f-e5465744050a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.140159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b9b99-335b-483e-ba89-997f0e6bf59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.140838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9b99-335b-483e-ba89-997f0e6bf59d admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.141118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9b99-335b-483e-ba89-997f0e6bf59d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.141389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b9b99-335b-483e-ba89-997f0e6bf59d admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.142283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b9b99-335b-483e-ba89-997f0e6bf59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.143916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b9b99-335b-483e-ba89-997f0e6bf59d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.151362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a265bdd5-7081-4294-ad73-c30024010710 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.153047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a265bdd5-7081-4294-ad73-c30024010710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.155095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 786/1555] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:24.156358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e92c-42f7-4642-a46c-3a30bc0bc819 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.156649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e92c-42f7-4642-a46c-3a30bc0bc819 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.156922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8e92c-42f7-4642-a46c-3a30bc0bc819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.161556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 770/1556] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.162842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f9eec-3904-4b38-ab38-58a0e06f41cb None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.163157 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f9eec-3904-4b38-ab38-58a0e06f41cb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.163496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f9eec-3904-4b38-ab38-58a0e06f41cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.176101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5aa8e92c-42f7-4642-a46c-3a30bc0bc819 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.177727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5aa8e92c-42f7-4642-a46c-3a30bc0bc819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.182370 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a51f9eec-3904-4b38-ab38-58a0e06f41cb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.183538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a51f9eec-3904-4b38-ab38-58a0e06f41cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.186413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 787/1557] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.187785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54c0d-6079-4986-a33c-2feee313cb77 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.188077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54c0d-6079-4986-a33c-2feee313cb77 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.188351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d54c0d-6079-4986-a33c-2feee313cb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.191834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 771/1558] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.192592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f9eec-3904-4b38-ab38-58a0e06f41cb None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.205010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f9eec-3904-4b38-ab38-58a0e06f41cb None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.207677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-53d54c0d-6079-4986-a33c-2feee313cb77 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.208891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-53d54c0d-6079-4986-a33c-2feee313cb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.209516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5476c-9895-4d6a-a8a8-7f31dae2dcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.210280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5476c-9895-4d6a-a8a8-7f31dae2dcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.210564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5476c-9895-4d6a-a8a8-7f31dae2dcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.210863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5476c-9895-4d6a-a8a8-7f31dae2dcd5 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.211803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5476c-9895-4d6a-a8a8-7f31dae2dcd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.212832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WARNING py.warnings [None req-2ec5476c-9895-4d6a-a8a8-7f31dae2dcd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:53:24.212832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: warnings.warn(msg) Jun 21 15:53:24.212832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: Jun 21 15:53:24.214088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5476c-9895-4d6a-a8a8-7f31dae2dcd5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.217564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 788/1559] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.218453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54c0d-6079-4986-a33c-2feee313cb77 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.229386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d54c0d-6079-4986-a33c-2feee313cb77 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.234082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.234747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.235026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.235296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.239347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 772/1560] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.239520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:24.239520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.239680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd42f-14b9-45cf-ae82-11bd02bcd080 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.239680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd42f-14b9-45cf-ae82-11bd02bcd080 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.239680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bd42f-14b9-45cf-ae82-11bd02bcd080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.239891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.247133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:24.253452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 789/1561] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:24.254538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.262548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2f3bd42f-14b9-45cf-ae82-11bd02bcd080 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.263927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2f3bd42f-14b9-45cf-ae82-11bd02bcd080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.265692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf2572-eb8b-45f8-8476-ead878abef9c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.270322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdecd26-a4f5-450e-99d0-198ac4d7eec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.271113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdecd26-a4f5-450e-99d0-198ac4d7eec8 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.271397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdecd26-a4f5-450e-99d0-198ac4d7eec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.271673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdecd26-a4f5-450e-99d0-198ac4d7eec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.272393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdecd26-a4f5-450e-99d0-198ac4d7eec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.272654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 773/1562] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.274274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b547d-5755-4c81-a922-fdef1574132e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.274619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b547d-5755-4c81-a922-fdef1574132e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.274855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b547d-5755-4c81-a922-fdef1574132e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.275649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdecd26-a4f5-450e-99d0-198ac4d7eec8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.281426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:24.289154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 790/1563] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:24.290657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdecd26-a4f5-450e-99d0-198ac4d7eec8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.295241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-352b547d-5755-4c81-a922-fdef1574132e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.296435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-352b547d-5755-4c81-a922-fdef1574132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.302220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdecd26-a4f5-450e-99d0-198ac4d7eec8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.304681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 774/1564] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.305637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b547d-5755-4c81-a922-fdef1574132e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.307438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.308329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.308824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.309097 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.309935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:24.310245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.312678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.315338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b547d-5755-4c81-a922-fdef1574132e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.319879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.320622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.320895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.321164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.321912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:24.322246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.323846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.326529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 791/1565] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:24.327980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.344517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 775/1566] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:24.344736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472a78-7b84-4543-b28d-132b5289db5e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.344847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.344951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.344951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.344951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.344951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.345674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:24.346014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.347649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.349721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c93dc8-63df-4130-a0c3-01b0704787c7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.354925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c2b21-2a48-432a-b988-ff559ee76b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.355611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2b21-2a48-432a-b988-ff559ee76b93 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.355913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2b21-2a48-432a-b988-ff559ee76b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.356184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2b21-2a48-432a-b988-ff559ee76b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.356905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c2b21-2a48-432a-b988-ff559ee76b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.358536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c2b21-2a48-432a-b988-ff559ee76b93 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.360705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 792/1567] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:24.362002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.372711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d652ea-b7c1-47ef-a1c4-0028dca526dd admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.377734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 776/1568] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:24.377734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c2b21-2a48-432a-b988-ff559ee76b93 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.380440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150463c-ca69-4935-8cae-825996b89d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.381111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150463c-ca69-4935-8cae-825996b89d22 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.381394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150463c-ca69-4935-8cae-825996b89d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.381677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150463c-ca69-4935-8cae-825996b89d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.382421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150463c-ca69-4935-8cae-825996b89d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.384354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150463c-ca69-4935-8cae-825996b89d22 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.391624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c2b21-2a48-432a-b988-ff559ee76b93 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.396646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.398415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.398415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.399102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.399537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:24.399756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.401360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.413343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 793/1569] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.416737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150463c-ca69-4935-8cae-825996b89d22 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.416860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 777/1570] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:24.416860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.432289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590586f-d948-486f-bae9-355a17eb9faf admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.492470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150463c-ca69-4935-8cae-825996b89d22 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.497531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.498487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.498968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.499167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.499297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] PATH_INFO: `/v3/projects/2281fb3898054e0881b900586871d5ab/users/f92d8614e59b45dfa6d264a46173690a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.499866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.500266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.500545 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.501347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:24.501710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.503308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.515066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2281fb3898054e0881b900586871d5ab, user_id=f92d8614e59b45dfa6d264a46173690a, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.515287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 778/1571] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:24.515968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.516146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING py.warnings [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:53:24.516146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: warnings.warn(msg) Jun 21 15:53:24.516146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: Jun 21 15:53:24.518158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.531820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 794/1572] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:24 2022] PUT /identity/v3/projects/2281fb3898054e0881b900586871d5ab/users/f92d8614e59b45dfa6d264a46173690a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:24.532762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.537534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824e189-c3b3-4c28-8ef3-bdd3750825dd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.601892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e356ff4-a020-48aa-9630-37146b28d42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.602573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e356ff4-a020-48aa-9630-37146b28d42b admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.602856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e356ff4-a020-48aa-9630-37146b28d42b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.603132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e356ff4-a020-48aa-9630-37146b28d42b admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.603827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e356ff4-a020-48aa-9630-37146b28d42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.605747 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e356ff4-a020-48aa-9630-37146b28d42b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.606575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053563c6-96de-4d3e-a0f5-a3b4f34f2e59 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.611495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fe0c0-801a-4110-814d-167db10c3f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.612307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fe0c0-801a-4110-814d-167db10c3f61 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.612570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fe0c0-801a-4110-814d-167db10c3f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.612846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fe0c0-801a-4110-814d-167db10c3f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.613546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fe0c0-801a-4110-814d-167db10c3f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.615534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7fe0c0-801a-4110-814d-167db10c3f61 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.634693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 779/1573] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.635649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e356ff4-a020-48aa-9630-37146b28d42b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.645081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 795/1574] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.646303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fe0c0-801a-4110-814d-167db10c3f61 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.660565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:24.690447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:24.715297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e356ff4-a020-48aa-9630-37146b28d42b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.720615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fe0c0-801a-4110-814d-167db10c3f61 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.720921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e211d-d661-47de-b137-81c4694e6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.721674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e211d-d661-47de-b137-81c4694e6ecf admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.721993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e211d-d661-47de-b137-81c4694e6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.722241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e211d-d661-47de-b137-81c4694e6ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.722965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e211d-d661-47de-b137-81c4694e6ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.724909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e211d-d661-47de-b137-81c4694e6ecf admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.726018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfb0a9-61d4-45fb-888a-e927b87c32d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.726736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfb0a9-61d4-45fb-888a-e927b87c32d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.727242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfb0a9-61d4-45fb-888a-e927b87c32d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.727528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfb0a9-61d4-45fb-888a-e927b87c32d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.728279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfb0a9-61d4-45fb-888a-e927b87c32d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.730057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfb0a9-61d4-45fb-888a-e927b87c32d0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.741161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 796/1575] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:24.742124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfb0a9-61d4-45fb-888a-e927b87c32d0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.755945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfb0a9-61d4-45fb-888a-e927b87c32d0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.758931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 780/1576] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.760398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e211d-d661-47de-b137-81c4694e6ecf admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.764672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab771b-c97b-41e9-a85e-52f298ba40a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.765401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab771b-c97b-41e9-a85e-52f298ba40a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.765718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab771b-c97b-41e9-a85e-52f298ba40a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.766012 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab771b-c97b-41e9-a85e-52f298ba40a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.766746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab771b-c97b-41e9-a85e-52f298ba40a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.768673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab771b-c97b-41e9-a85e-52f298ba40a1 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.791152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 797/1577] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/users => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.792235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab771b-c97b-41e9-a85e-52f298ba40a1 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.802792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab771b-c97b-41e9-a85e-52f298ba40a1 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.829786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e211d-d661-47de-b137-81c4694e6ecf admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.834613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d216127-e950-43d7-bd92-d0b15e00086d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.835362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216127-e950-43d7-bd92-d0b15e00086d admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.835647 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216127-e950-43d7-bd92-d0b15e00086d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.835925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216127-e950-43d7-bd92-d0b15e00086d admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.836687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d216127-e950-43d7-bd92-d0b15e00086d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.838570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d216127-e950-43d7-bd92-d0b15e00086d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.864191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065d260-f515-456e-a638-9f957b8790c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.864902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065d260-f515-456e-a638-9f957b8790c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.865181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065d260-f515-456e-a638-9f957b8790c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.865457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065d260-f515-456e-a638-9f957b8790c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.866239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065d260-f515-456e-a638-9f957b8790c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.867547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 781/1578] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.868700 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78ddaf-3c1e-4af1-89c9-7dea14356a75 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.868826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065d260-f515-456e-a638-9f957b8790c0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.869513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78ddaf-3c1e-4af1-89c9-7dea14356a75 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.869513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78ddaf-3c1e-4af1-89c9-7dea14356a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.896896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2a78ddaf-3c1e-4af1-89c9-7dea14356a75 None None] MFA Rules not processed for user `f92d8614e59b45dfa6d264a46173690a`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:24.898188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2a78ddaf-3c1e-4af1-89c9-7dea14356a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:24.898962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 798/1579] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:24.899787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065d260-f515-456e-a638-9f957b8790c0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.966489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065d260-f515-456e-a638-9f957b8790c0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.967337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 782/1580] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:24.967978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78ddaf-3c1e-4af1-89c9-7dea14356a75 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:24.974937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f8619-d791-4d5b-8103-b54c1efd0608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.975647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f8619-d791-4d5b-8103-b54c1efd0608 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.975953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f8619-d791-4d5b-8103-b54c1efd0608 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.976310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f8619-d791-4d5b-8103-b54c1efd0608 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.976953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f8619-d791-4d5b-8103-b54c1efd0608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.978977 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f8619-d791-4d5b-8103-b54c1efd0608 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.982703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78ddaf-3c1e-4af1-89c9-7dea14356a75 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:24.989814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4fa42-4461-4042-88ab-37e626333248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:24.990793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4fa42-4461-4042-88ab-37e626333248 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:24.991362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4fa42-4461-4042-88ab-37e626333248 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:24.991911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4fa42-4461-4042-88ab-37e626333248 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:24.992888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4fa42-4461-4042-88ab-37e626333248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:24.995808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4fa42-4461-4042-88ab-37e626333248 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:24.996667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:25.006322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 799/1581] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.006322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f8619-d791-4d5b-8103-b54c1efd0608 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.016605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f8619-d791-4d5b-8103-b54c1efd0608 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.018057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 783/1582] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:24 2022] POST /identity/v3/users => generated 302 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.019041 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4fa42-4461-4042-88ab-37e626333248 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.021460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89911ce4-ccd1-4925-84e9-cbc42dc332d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.022398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911ce4-ccd1-4925-84e9-cbc42dc332d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.022734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911ce4-ccd1-4925-84e9-cbc42dc332d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.023118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911ce4-ccd1-4925-84e9-cbc42dc332d8 admin admin] PATH_INFO: `/v3/projects/3be3a5d422414cb490c344af063afc60/users/22b55c4684194f2cb19cf31edd9e7adc/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.029628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4fa42-4461-4042-88ab-37e626333248 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.034718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604115d-04ad-4e42-8b3f-ac6591b121b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.035615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604115d-04ad-4e42-8b3f-ac6591b121b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.035906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604115d-04ad-4e42-8b3f-ac6591b121b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.036187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604115d-04ad-4e42-8b3f-ac6591b121b2 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.036893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604115d-04ad-4e42-8b3f-ac6591b121b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.038857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89911ce4-ccd1-4925-84e9-cbc42dc332d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be3a5d422414cb490c344af063afc60, user_id=22b55c4684194f2cb19cf31edd9e7adc, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.039667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604115d-04ad-4e42-8b3f-ac6591b121b2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.041452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89911ce4-ccd1-4925-84e9-cbc42dc332d8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.057540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 800/1583] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:25 2022] PUT /identity/v3/projects/3be3a5d422414cb490c344af063afc60/users/22b55c4684194f2cb19cf31edd9e7adc/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:25.058447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89911ce4-ccd1-4925-84e9-cbc42dc332d8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.067428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 784/1584] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/users => generated 291 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.068572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604115d-04ad-4e42-8b3f-ac6591b121b2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.077273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:25.085988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604115d-04ad-4e42-8b3f-ac6591b121b2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.154262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89911ce4-ccd1-4925-84e9-cbc42dc332d8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.159505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b2443-8e2f-4f05-99a8-71f054078eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.160446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b2443-8e2f-4f05-99a8-71f054078eac admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.160734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b2443-8e2f-4f05-99a8-71f054078eac admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.161007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b2443-8e2f-4f05-99a8-71f054078eac admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.161915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b2443-8e2f-4f05-99a8-71f054078eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.163695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b2443-8e2f-4f05-99a8-71f054078eac admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.163834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49051fc4-91d9-45cf-bcb2-598bbc121a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.164496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49051fc4-91d9-45cf-bcb2-598bbc121a36 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.164767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49051fc4-91d9-45cf-bcb2-598bbc121a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.165044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49051fc4-91d9-45cf-bcb2-598bbc121a36 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.165750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49051fc4-91d9-45cf-bcb2-598bbc121a36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.167732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49051fc4-91d9-45cf-bcb2-598bbc121a36 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.174447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 801/1585] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.175558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b2443-8e2f-4f05-99a8-71f054078eac admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.186791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b2443-8e2f-4f05-99a8-71f054078eac admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.191977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 785/1586] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.191977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49051fc4-91d9-45cf-bcb2-598bbc121a36 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.194887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976e1c1-7fe0-4986-9abb-ca1e14047df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.195672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976e1c1-7fe0-4986-9abb-ca1e14047df9 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.195956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976e1c1-7fe0-4986-9abb-ca1e14047df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.196238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976e1c1-7fe0-4986-9abb-ca1e14047df9 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.196943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976e1c1-7fe0-4986-9abb-ca1e14047df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.198925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976e1c1-7fe0-4986-9abb-ca1e14047df9 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.200044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49051fc4-91d9-45cf-bcb2-598bbc121a36 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.204916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f392-109c-4660-b4da-8519aba0bad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.205585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f392-109c-4660-b4da-8519aba0bad1 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.205899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f392-109c-4660-b4da-8519aba0bad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.206179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f392-109c-4660-b4da-8519aba0bad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.206877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82f392-109c-4660-b4da-8519aba0bad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.208786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82f392-109c-4660-b4da-8519aba0bad1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.223175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 802/1587] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.224408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976e1c1-7fe0-4986-9abb-ca1e14047df9 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.236750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976e1c1-7fe0-4986-9abb-ca1e14047df9 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.243442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f5127-9b56-4767-9158-20e4945b94a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.244212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f5127-9b56-4767-9158-20e4945b94a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.244574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f5127-9b56-4767-9158-20e4945b94a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.244968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f5127-9b56-4767-9158-20e4945b94a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.245794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f5127-9b56-4767-9158-20e4945b94a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.246577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 786/1588] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.247515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f392-109c-4660-b4da-8519aba0bad1 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.247664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f5127-9b56-4767-9158-20e4945b94a2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.263698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 803/1589] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.263698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b7ba6-2920-4241-8720-d55b259bacb6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.263698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b7ba6-2920-4241-8720-d55b259bacb6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.263698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b7ba6-2920-4241-8720-d55b259bacb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.300129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-137b7ba6-2920-4241-8720-d55b259bacb6 None None] MFA Rules not processed for user `22b55c4684194f2cb19cf31edd9e7adc`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:25.302134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-137b7ba6-2920-4241-8720-d55b259bacb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:25.318067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f392-109c-4660-b4da-8519aba0bad1 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.325002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cafa29-004c-4116-9498-270c1deea8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.325687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cafa29-004c-4116-9498-270c1deea8da admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.325993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cafa29-004c-4116-9498-270c1deea8da admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.326267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cafa29-004c-4116-9498-270c1deea8da admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.327002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cafa29-004c-4116-9498-270c1deea8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.328599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cafa29-004c-4116-9498-270c1deea8da admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.339839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 787/1590] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.340676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cafa29-004c-4116-9498-270c1deea8da admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.350502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cafa29-004c-4116-9498-270c1deea8da admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.356111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b46558-a755-45e6-a3c7-511fa8abeda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.356775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46558-a755-45e6-a3c7-511fa8abeda8 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.357039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46558-a755-45e6-a3c7-511fa8abeda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.357312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b46558-a755-45e6-a3c7-511fa8abeda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.358068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b46558-a755-45e6-a3c7-511fa8abeda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.359642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b46558-a755-45e6-a3c7-511fa8abeda8 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.370118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 804/1591] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:25.371049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b7ba6-2920-4241-8720-d55b259bacb6 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.373969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 788/1592] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.374933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b46558-a755-45e6-a3c7-511fa8abeda8 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.383217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b7ba6-2920-4241-8720-d55b259bacb6 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.385236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b46558-a755-45e6-a3c7-511fa8abeda8 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.388062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e974c84-6147-4533-ad64-78381b923e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.388621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974c84-6147-4533-ad64-78381b923e7e admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.388949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974c84-6147-4533-ad64-78381b923e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.389138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974c84-6147-4533-ad64-78381b923e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.389899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e974c84-6147-4533-ad64-78381b923e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.390086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5455b-9802-4515-9212-ba163299f94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.390713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5455b-9802-4515-9212-ba163299f94a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.390989 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5455b-9802-4515-9212-ba163299f94a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.391272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5455b-9802-4515-9212-ba163299f94a admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.391502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e974c84-6147-4533-ad64-78381b923e7e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.391996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5455b-9802-4515-9212-ba163299f94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.393584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5455b-9802-4515-9212-ba163299f94a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.403029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 805/1593] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.404027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e974c84-6147-4533-ad64-78381b923e7e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.407599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 789/1594] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.407599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5455b-9802-4515-9212-ba163299f94a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.417006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5455b-9802-4515-9212-ba163299f94a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.417185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e974c84-6147-4533-ad64-78381b923e7e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.421961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c06ee-4e98-4516-a5b1-2fac17a001ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.422370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5d574-7816-4eb4-be3a-bf7e3aacb74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.422623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c06ee-4e98-4516-a5b1-2fac17a001ae admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.422901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c06ee-4e98-4516-a5b1-2fac17a001ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.423034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d574-7816-4eb4-be3a-bf7e3aacb74a admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.423173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c06ee-4e98-4516-a5b1-2fac17a001ae admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.423317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d574-7816-4eb4-be3a-bf7e3aacb74a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.423590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d574-7816-4eb4-be3a-bf7e3aacb74a admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.423858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c06ee-4e98-4516-a5b1-2fac17a001ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.424326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5d574-7816-4eb4-be3a-bf7e3aacb74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.425762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c06ee-4e98-4516-a5b1-2fac17a001ae admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.425943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5d574-7816-4eb4-be3a-bf7e3aacb74a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.437841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 806/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.438780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5d574-7816-4eb4-be3a-bf7e3aacb74a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.449282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 790/1596] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.449477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5d574-7816-4eb4-be3a-bf7e3aacb74a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.450287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c06ee-4e98-4516-a5b1-2fac17a001ae admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.454395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8abd1-ca25-4148-ba5c-b605f5e522c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.455111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8abd1-ca25-4148-ba5c-b605f5e522c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.455404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8abd1-ca25-4148-ba5c-b605f5e522c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.455713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8abd1-ca25-4148-ba5c-b605f5e522c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.456466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8abd1-ca25-4148-ba5c-b605f5e522c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.458153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8abd1-ca25-4148-ba5c-b605f5e522c4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.460135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c06ee-4e98-4516-a5b1-2fac17a001ae admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.464952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:25.466025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113c0c3-e302-48fb-955d-baf05c49b0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.467003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113c0c3-e302-48fb-955d-baf05c49b0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.467273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113c0c3-e302-48fb-955d-baf05c49b0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.467538 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113c0c3-e302-48fb-955d-baf05c49b0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.468323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113c0c3-e302-48fb-955d-baf05c49b0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.470087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113c0c3-e302-48fb-955d-baf05c49b0b0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.471613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 807/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.473175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8abd1-ca25-4148-ba5c-b605f5e522c4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.482619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 791/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.486855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113c0c3-e302-48fb-955d-baf05c49b0b0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.486972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8abd1-ca25-4148-ba5c-b605f5e522c4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.491952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc6bea-1340-4925-ab7d-1483e87e234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.492695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6bea-1340-4925-ab7d-1483e87e234d admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.493049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6bea-1340-4925-ab7d-1483e87e234d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.493396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc6bea-1340-4925-ab7d-1483e87e234d admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.494101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113c0c3-e302-48fb-955d-baf05c49b0b0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.494226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc6bea-1340-4925-ab7d-1483e87e234d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.496149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc6bea-1340-4925-ab7d-1483e87e234d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.498698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f269d2e-9a60-4f03-828b-31657502dafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.499344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f269d2e-9a60-4f03-828b-31657502dafe admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.499622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f269d2e-9a60-4f03-828b-31657502dafe admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.499893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f269d2e-9a60-4f03-828b-31657502dafe admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.500610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f269d2e-9a60-4f03-828b-31657502dafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.502536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f269d2e-9a60-4f03-828b-31657502dafe admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.514606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 792/1599] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.518518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f269d2e-9a60-4f03-828b-31657502dafe admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.523785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 808/1600] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.524809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc6bea-1340-4925-ab7d-1483e87e234d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.525484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:25.529628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f269d2e-9a60-4f03-828b-31657502dafe admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.593511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc6bea-1340-4925-ab7d-1483e87e234d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.595113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be08f3-2f3d-486a-aba3-323408ec9022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.596023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be08f3-2f3d-486a-aba3-323408ec9022 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.596350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be08f3-2f3d-486a-aba3-323408ec9022 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.596564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be08f3-2f3d-486a-aba3-323408ec9022 admin admin] PATH_INFO: `/v3/projects/74f35c94371b44c6936b1e713a398f1c/users/fa5a48626c934dc2aa4f24002970b6d5/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.598980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91bd7d-3107-4798-a175-1fbf9a4a19d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.599717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91bd7d-3107-4798-a175-1fbf9a4a19d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.600004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91bd7d-3107-4798-a175-1fbf9a4a19d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.600283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91bd7d-3107-4798-a175-1fbf9a4a19d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.601083 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91bd7d-3107-4798-a175-1fbf9a4a19d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.602872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91bd7d-3107-4798-a175-1fbf9a4a19d5 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.616675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 809/1601] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.616899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be08f3-2f3d-486a-aba3-323408ec9022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f35c94371b44c6936b1e713a398f1c, user_id=fa5a48626c934dc2aa4f24002970b6d5, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.617021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91bd7d-3107-4798-a175-1fbf9a4a19d5 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.617115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be08f3-2f3d-486a-aba3-323408ec9022 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.625653 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91bd7d-3107-4798-a175-1fbf9a4a19d5 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.633176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 793/1602] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:25 2022] PUT /identity/v3/projects/74f35c94371b44c6936b1e713a398f1c/users/fa5a48626c934dc2aa4f24002970b6d5/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:25.634047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be08f3-2f3d-486a-aba3-323408ec9022 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.687032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85297ddc-d6d1-4752-84c1-26ef52d6407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.687708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297ddc-d6d1-4752-84c1-26ef52d6407b admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.687993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297ddc-d6d1-4752-84c1-26ef52d6407b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.688265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85297ddc-d6d1-4752-84c1-26ef52d6407b admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.689123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85297ddc-d6d1-4752-84c1-26ef52d6407b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.690764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85297ddc-d6d1-4752-84c1-26ef52d6407b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.701654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 810/1603] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.702647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85297ddc-d6d1-4752-84c1-26ef52d6407b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.703601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be08f3-2f3d-486a-aba3-323408ec9022 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.709918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aefdd-30c2-4516-9f75-3a01e5ba1a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.710656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aefdd-30c2-4516-9f75-3a01e5ba1a0c admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.711114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aefdd-30c2-4516-9f75-3a01e5ba1a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.711246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aefdd-30c2-4516-9f75-3a01e5ba1a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.711905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aefdd-30c2-4516-9f75-3a01e5ba1a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.713515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aefdd-30c2-4516-9f75-3a01e5ba1a0c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.716404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85297ddc-d6d1-4752-84c1-26ef52d6407b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.721250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9e76b-ed32-4264-b133-cf9c454b3a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.722184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9e76b-ed32-4264-b133-cf9c454b3a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.722463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9e76b-ed32-4264-b133-cf9c454b3a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.722735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9e76b-ed32-4264-b133-cf9c454b3a36 admin admin] PATH_INFO: `/v3/projects/8a626149a0ee4b3db0113c7084b37cfa/users/dd95af9b9bc246d7aa6445e25e39a7db/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.727635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 794/1604] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:25.727635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aefdd-30c2-4516-9f75-3a01e5ba1a0c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.734909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aefdd-30c2-4516-9f75-3a01e5ba1a0c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.741129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9e76b-ed32-4264-b133-cf9c454b3a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a626149a0ee4b3db0113c7084b37cfa, user_id=dd95af9b9bc246d7aa6445e25e39a7db, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.741512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b8b63-c1de-4389-99a8-44eec3de4744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.742691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b8b63-c1de-4389-99a8-44eec3de4744 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.742976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b8b63-c1de-4389-99a8-44eec3de4744 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.743257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b8b63-c1de-4389-99a8-44eec3de4744 admin admin] PATH_INFO: `/v3/projects/0a8326517d0f4bc191b63059f6b5721b/users/76f88f8b791743e3a6b7cb0d29f3b4f6/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.744130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9e76b-ed32-4264-b133-cf9c454b3a36 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.760644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 811/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:25 2022] PUT /identity/v3/projects/8a626149a0ee4b3db0113c7084b37cfa/users/dd95af9b9bc246d7aa6445e25e39a7db/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:25.761456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9e76b-ed32-4264-b133-cf9c454b3a36 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.768471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b8b63-c1de-4389-99a8-44eec3de4744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8326517d0f4bc191b63059f6b5721b, user_id=76f88f8b791743e3a6b7cb0d29f3b4f6, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.771689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b8b63-c1de-4389-99a8-44eec3de4744 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.785757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 795/1606] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:25 2022] PUT /identity/v3/projects/0a8326517d0f4bc191b63059f6b5721b/users/76f88f8b791743e3a6b7cb0d29f3b4f6/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:25.786746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b8b63-c1de-4389-99a8-44eec3de4744 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.823579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:25.842175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9e76b-ed32-4264-b133-cf9c454b3a36 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.858889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b8b63-c1de-4389-99a8-44eec3de4744 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.866296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599069a3-05ca-433c-a1b5-868272738744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.867229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599069a3-05ca-433c-a1b5-868272738744 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.867767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599069a3-05ca-433c-a1b5-868272738744 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.868220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599069a3-05ca-433c-a1b5-868272738744 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.869440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599069a3-05ca-433c-a1b5-868272738744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.872647 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599069a3-05ca-433c-a1b5-868272738744 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.883762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:25.903431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 796/1607] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/users => generated 309 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:25.904715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599069a3-05ca-433c-a1b5-868272738744 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.915635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599069a3-05ca-433c-a1b5-868272738744 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:25.921946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8005cb2-72b2-46d0-96d8-4949a71c0a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.922892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8005cb2-72b2-46d0-96d8-4949a71c0a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.923238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8005cb2-72b2-46d0-96d8-4949a71c0a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.923519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8005cb2-72b2-46d0-96d8-4949a71c0a32 admin admin] PATH_INFO: `/v3/projects/74f3f5c4b63c45d691e2b66883eda54a/users/75407a21b1fd46e8ba10540fc5a4c30a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.924763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba8a16-bb7a-4712-93a1-620de82bcad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:25.925668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba8a16-bb7a-4712-93a1-620de82bcad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.925991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba8a16-bb7a-4712-93a1-620de82bcad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.926270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba8a16-bb7a-4712-93a1-620de82bcad4 admin admin] PATH_INFO: `/v3/projects/d20d29454c00499282eb86547eeb4c10/users/9a67a5c4b7034d22b78331b907cc20bc/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.940810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8005cb2-72b2-46d0-96d8-4949a71c0a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f3f5c4b63c45d691e2b66883eda54a, user_id=75407a21b1fd46e8ba10540fc5a4c30a, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.942080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba8a16-bb7a-4712-93a1-620de82bcad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20d29454c00499282eb86547eeb4c10, user_id=9a67a5c4b7034d22b78331b907cc20bc, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:25.944091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8005cb2-72b2-46d0-96d8-4949a71c0a32 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.944369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba8a16-bb7a-4712-93a1-620de82bcad4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:25.958789 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 812/1608] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:25 2022] PUT /identity/v3/projects/d20d29454c00499282eb86547eeb4c10/users/9a67a5c4b7034d22b78331b907cc20bc/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:25.963796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 797/1609] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:25 2022] PUT /identity/v3/projects/74f3f5c4b63c45d691e2b66883eda54a/users/75407a21b1fd46e8ba10540fc5a4c30a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:25.963994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe1516-6f5c-4ba1-8239-2e2a852336aa None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:25.963994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe1516-6f5c-4ba1-8239-2e2a852336aa None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:25.963994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe1516-6f5c-4ba1-8239-2e2a852336aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:25.964348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8005cb2-72b2-46d0-96d8-4949a71c0a32 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:25.979697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-7cbe1516-6f5c-4ba1-8239-2e2a852336aa None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:25.980928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7cbe1516-6f5c-4ba1-8239-2e2a852336aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:26.031407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8005cb2-72b2-46d0-96d8-4949a71c0a32 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.037122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5efb1-226a-4ff6-8408-042ff344f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.038089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5efb1-226a-4ff6-8408-042ff344f8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.038404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5efb1-226a-4ff6-8408-042ff344f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.039120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5efb1-226a-4ff6-8408-042ff344f8e2 admin admin] PATH_INFO: `/v3/projects/707b41a2e0ce4a8abc4e26c88ce94267/users/23031b94dc7f47bf8272b266cbe18a8c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.042852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 813/1610] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:26.043639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe1516-6f5c-4ba1-8239-2e2a852336aa None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.055159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe1516-6f5c-4ba1-8239-2e2a852336aa None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.056849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5efb1-226a-4ff6-8408-042ff344f8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=707b41a2e0ce4a8abc4e26c88ce94267, user_id=23031b94dc7f47bf8272b266cbe18a8c, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.059552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5efb1-226a-4ff6-8408-042ff344f8e2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.060784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bb549-cc4a-4d9a-bffa-796a1046bb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.061814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb549-cc4a-4d9a-bffa-796a1046bb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.062185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb549-cc4a-4d9a-bffa-796a1046bb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.062518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bb549-cc4a-4d9a-bffa-796a1046bb95 admin admin] PATH_INFO: `/v3/projects/1675afbdcaec408e8d7049a1932c9690/users/212f41d7d26b4ed1810c6672dfa360db/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.074687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 798/1611] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:25 2022] PUT /identity/v3/projects/707b41a2e0ce4a8abc4e26c88ce94267/users/23031b94dc7f47bf8272b266cbe18a8c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:26.076238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946d76b-96e8-43f6-9370-fa82962101da None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.076513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946d76b-96e8-43f6-9370-fa82962101da None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.076938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946d76b-96e8-43f6-9370-fa82962101da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.083946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bb549-cc4a-4d9a-bffa-796a1046bb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1675afbdcaec408e8d7049a1932c9690, user_id=212f41d7d26b4ed1810c6672dfa360db, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.086781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866bb549-cc4a-4d9a-bffa-796a1046bb95 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.102404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 814/1612] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:26 2022] PUT /identity/v3/projects/1675afbdcaec408e8d7049a1932c9690/users/212f41d7d26b4ed1810c6672dfa360db/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:26.103947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b77e3-5942-4744-b624-889124717b79 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.104062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b77e3-5942-4744-b624-889124717b79 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.104336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b77e3-5942-4744-b624-889124717b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.106433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2946d76b-96e8-43f6-9370-fa82962101da None None] MFA Rules not processed for user `dd95af9b9bc246d7aa6445e25e39a7db`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:26.107703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2946d76b-96e8-43f6-9370-fa82962101da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:26.122244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-035b77e3-5942-4744-b624-889124717b79 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:26.123435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-035b77e3-5942-4744-b624-889124717b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:26.174955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:26.181191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 799/1613] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:26.190761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946d76b-96e8-43f6-9370-fa82962101da None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.190959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 815/1614] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:26.196830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a450e-a8db-43f0-8ace-3c799f0b937d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.197120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a450e-a8db-43f0-8ace-3c799f0b937d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.197407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a450e-a8db-43f0-8ace-3c799f0b937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.205551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946d76b-96e8-43f6-9370-fa82962101da None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.214853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3e087-5cda-49b0-b35f-3cb74e52d653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.215637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3e087-5cda-49b0-b35f-3cb74e52d653 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.215812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3e087-5cda-49b0-b35f-3cb74e52d653 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.216124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3e087-5cda-49b0-b35f-3cb74e52d653 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.216839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3e087-5cda-49b0-b35f-3cb74e52d653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.218587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3e087-5cda-49b0-b35f-3cb74e52d653 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.224411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b90a450e-a8db-43f0-8ace-3c799f0b937d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:26.225671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b90a450e-a8db-43f0-8ace-3c799f0b937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:26.235451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 800/1615] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:26.236727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c14d6e-3a88-44da-ae33-8cdb59e63b43 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.237021 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c14d6e-3a88-44da-ae33-8cdb59e63b43 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.237303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c14d6e-3a88-44da-ae33-8cdb59e63b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.249792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 816/1616] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:26.250403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a450e-a8db-43f0-8ace-3c799f0b937d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.259080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-85c14d6e-3a88-44da-ae33-8cdb59e63b43 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:26.260365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-85c14d6e-3a88-44da-ae33-8cdb59e63b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:26.269161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 801/1617] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:26.270747 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4db1-89f3-4f3d-be56-672290348057 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.271035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4db1-89f3-4f3d-be56-672290348057 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.271333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e4db1-89f3-4f3d-be56-672290348057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.290766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0f2e4db1-89f3-4f3d-be56-672290348057 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:26.291876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0f2e4db1-89f3-4f3d-be56-672290348057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:26.299521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 802/1618] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:26.300318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e4db1-89f3-4f3d-be56-672290348057 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.313564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e4db1-89f3-4f3d-be56-672290348057 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.313755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a450e-a8db-43f0-8ace-3c799f0b937d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.317419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4f459-b606-44ab-a4aa-de35d588c651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.318107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4f459-b606-44ab-a4aa-de35d588c651 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.318385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4f459-b606-44ab-a4aa-de35d588c651 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.318660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4f459-b606-44ab-a4aa-de35d588c651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.318777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9729e64-1419-42f3-8cd0-48bbc5e78866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.319573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729e64-1419-42f3-8cd0-48bbc5e78866 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.319950 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729e64-1419-42f3-8cd0-48bbc5e78866 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.320295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9729e64-1419-42f3-8cd0-48bbc5e78866 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.321207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9729e64-1419-42f3-8cd0-48bbc5e78866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.322925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9729e64-1419-42f3-8cd0-48bbc5e78866 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.329161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4f459-b606-44ab-a4aa-de35d588c651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.330628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4f459-b606-44ab-a4aa-de35d588c651 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.340638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 817/1619] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:26.341617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4f459-b606-44ab-a4aa-de35d588c651 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.345284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 803/1620] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:26.346958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5778212-d90f-4e7e-925f-f2225640b1bd None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.347250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5778212-d90f-4e7e-925f-f2225640b1bd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.347528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5778212-d90f-4e7e-925f-f2225640b1bd None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.349283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 804/1621] 10.222.0.52 () {54 vars in 961 bytes} [Tue Jun 21 15:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:53:26.350241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5778212-d90f-4e7e-925f-f2225640b1bd None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.354025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4f459-b606-44ab-a4aa-de35d588c651 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.357897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:26.359336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771ecea-7855-440b-9b70-3eea0e4c0371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.360170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771ecea-7855-440b-9b70-3eea0e4c0371 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.360506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771ecea-7855-440b-9b70-3eea0e4c0371 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.361173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771ecea-7855-440b-9b70-3eea0e4c0371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.368245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5778212-d90f-4e7e-925f-f2225640b1bd None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.375261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c9760-9595-4d20-89cb-937946a3dada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.375624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771ecea-7855-440b-9b70-3eea0e4c0371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.377064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c9760-9595-4d20-89cb-937946a3dada admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.377195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771ecea-7855-440b-9b70-3eea0e4c0371 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.377616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c9760-9595-4d20-89cb-937946a3dada admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.378083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c9760-9595-4d20-89cb-937946a3dada admin admin] PATH_INFO: `/v3/projects/97f65c3678af4f3eba2297ba89936387/users/abd5faecbc39415c884a8412f7a26c84/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.388148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 818/1622] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:26.388906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771ecea-7855-440b-9b70-3eea0e4c0371 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.398039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771ecea-7855-440b-9b70-3eea0e4c0371 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.401987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df345210-49c4-4bff-bfa4-98c10a16f59d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.402740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c9760-9595-4d20-89cb-937946a3dada admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f65c3678af4f3eba2297ba89936387, user_id=abd5faecbc39415c884a8412f7a26c84, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.402872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df345210-49c4-4bff-bfa4-98c10a16f59d service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.403069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df345210-49c4-4bff-bfa4-98c10a16f59d service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.403299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df345210-49c4-4bff-bfa4-98c10a16f59d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.406664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c9760-9595-4d20-89cb-937946a3dada admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.414475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df345210-49c4-4bff-bfa4-98c10a16f59d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.416215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df345210-49c4-4bff-bfa4-98c10a16f59d service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.423548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 805/1623] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:26 2022] PUT /identity/v3/projects/97f65c3678af4f3eba2297ba89936387/users/abd5faecbc39415c884a8412f7a26c84/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:26.425302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c9760-9595-4d20-89cb-937946a3dada admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.427047 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 819/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:26.427898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df345210-49c4-4bff-bfa4-98c10a16f59d service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.437262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df345210-49c4-4bff-bfa4-98c10a16f59d service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.499162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf218f51-bfa1-4f19-9139-c28bbc7f1aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.499532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c9760-9595-4d20-89cb-937946a3dada admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.499941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf218f51-bfa1-4f19-9139-c28bbc7f1aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.500176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf218f51-bfa1-4f19-9139-c28bbc7f1aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.500452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf218f51-bfa1-4f19-9139-c28bbc7f1aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.505507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818e3d1-441b-4458-8cb1-ff1468a697bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.506821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e3d1-441b-4458-8cb1-ff1468a697bf service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.508003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e3d1-441b-4458-8cb1-ff1468a697bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.508630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e3d1-441b-4458-8cb1-ff1468a697bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.576844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf218f51-bfa1-4f19-9139-c28bbc7f1aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.578407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf218f51-bfa1-4f19-9139-c28bbc7f1aa5 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.588206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818e3d1-441b-4458-8cb1-ff1468a697bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.590952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818e3d1-441b-4458-8cb1-ff1468a697bf service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.591519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 820/1625] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:26.592272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf218f51-bfa1-4f19-9139-c28bbc7f1aa5 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.602153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf218f51-bfa1-4f19-9139-c28bbc7f1aa5 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.607105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03be74-00c0-4864-b3e4-2a6aaf6d1c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.608014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03be74-00c0-4864-b3e4-2a6aaf6d1c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.608348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03be74-00c0-4864-b3e4-2a6aaf6d1c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.608630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03be74-00c0-4864-b3e4-2a6aaf6d1c61 admin admin] PATH_INFO: `/v3/projects/1675afbdcaec408e8d7049a1932c9690/users/212f41d7d26b4ed1810c6672dfa360db/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.610729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03be74-00c0-4864-b3e4-2a6aaf6d1c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1675afbdcaec408e8d7049a1932c9690, user_id=212f41d7d26b4ed1810c6672dfa360db, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.611844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 806/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:26.613939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03be74-00c0-4864-b3e4-2a6aaf6d1c61 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.614660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2079-3168-4177-9738-38fd8ccb1eb5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.615251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2079-3168-4177-9738-38fd8ccb1eb5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.615838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2079-3168-4177-9738-38fd8ccb1eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.621700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:26.638082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 821/1627] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:26 2022] PUT /identity/v3/projects/1675afbdcaec408e8d7049a1932c9690/users/212f41d7d26b4ed1810c6672dfa360db/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:26.639608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03be74-00c0-4864-b3e4-2a6aaf6d1c61 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.646600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-fd5e2079-3168-4177-9738-38fd8ccb1eb5 None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:26.648175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fd5e2079-3168-4177-9738-38fd8ccb1eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:26.757765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 807/1628] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Jun 21 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:26.761963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03be74-00c0-4864-b3e4-2a6aaf6d1c61 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.767559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0a5d-32cf-407e-bbe0-b240bd6696f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.768782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300079-f795-4f32-8bb6-739675b686e9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.768782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300079-f795-4f32-8bb6-739675b686e9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.768977 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0a5d-32cf-407e-bbe0-b240bd6696f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.769079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300079-f795-4f32-8bb6-739675b686e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.769394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0a5d-32cf-407e-bbe0-b240bd6696f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.769758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df0a5d-32cf-407e-bbe0-b240bd6696f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.771216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df0a5d-32cf-407e-bbe0-b240bd6696f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.776517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df0a5d-32cf-407e-bbe0-b240bd6696f3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.796316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 822/1629] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:26.797720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0a5d-32cf-407e-bbe0-b240bd6696f3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.816485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df0a5d-32cf-407e-bbe0-b240bd6696f3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.820804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7d300079-f795-4f32-8bb6-739675b686e9 None None] MFA Rules not processed for user `212f41d7d26b4ed1810c6672dfa360db`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:26.822249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7d300079-f795-4f32-8bb6-739675b686e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:26.823751 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a96e-6dc7-4b09-8dd8-f29f9d86b469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.824578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a96e-6dc7-4b09-8dd8-f29f9d86b469 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.824924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a96e-6dc7-4b09-8dd8-f29f9d86b469 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.825268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a96e-6dc7-4b09-8dd8-f29f9d86b469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.837546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:26.903213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a96e-6dc7-4b09-8dd8-f29f9d86b469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.905698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a96e-6dc7-4b09-8dd8-f29f9d86b469 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.919065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 823/1630] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:26.920007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a96e-6dc7-4b09-8dd8-f29f9d86b469 service nova] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:26.933286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a96e-6dc7-4b09-8dd8-f29f9d86b469 service nova] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:26.938881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae546c-4829-4489-bd3d-e06d4ff8bfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:26.940304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae546c-4829-4489-bd3d-e06d4ff8bfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.940764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae546c-4829-4489-bd3d-e06d4ff8bfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.941112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae546c-4829-4489-bd3d-e06d4ff8bfa6 admin admin] PATH_INFO: `/v3/projects/97f65c3678af4f3eba2297ba89936387/users/abd5faecbc39415c884a8412f7a26c84/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:26.943373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae546c-4829-4489-bd3d-e06d4ff8bfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f65c3678af4f3eba2297ba89936387, user_id=abd5faecbc39415c884a8412f7a26c84, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:26.947069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae546c-4829-4489-bd3d-e06d4ff8bfa6 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:26.971340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 824/1631] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:26 2022] PUT /identity/v3/projects/97f65c3678af4f3eba2297ba89936387/users/abd5faecbc39415c884a8412f7a26c84/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:26.973754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189efa5-26a4-451d-9292-55548b75af63 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:26.974076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189efa5-26a4-451d-9292-55548b75af63 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:26.974342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189efa5-26a4-451d-9292-55548b75af63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:27.011944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f189efa5-26a4-451d-9292-55548b75af63 None None] MFA Rules not processed for user `abd5faecbc39415c884a8412f7a26c84`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:27.013434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f189efa5-26a4-451d-9292-55548b75af63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:27.062754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 808/1632] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:27.081115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d300079-f795-4f32-8bb6-739675b686e9 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:27.122335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 825/1633] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:27.132309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189efa5-26a4-451d-9292-55548b75af63 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:27.144639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:27.170860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d300079-f795-4f32-8bb6-739675b686e9 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:27.220629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189efa5-26a4-451d-9292-55548b75af63 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:27.227756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1431d8-1255-443c-89f1-5960308f835d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:27.228849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1431d8-1255-443c-89f1-5960308f835d service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:27.229358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1431d8-1255-443c-89f1-5960308f835d service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:27.230624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1431d8-1255-443c-89f1-5960308f835d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:27.231872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c85690-8d7c-4d07-b681-740f6d01e306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:27.232816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c85690-8d7c-4d07-b681-740f6d01e306 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:27.233261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c85690-8d7c-4d07-b681-740f6d01e306 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:27.233844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c85690-8d7c-4d07-b681-740f6d01e306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:27.246275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1431d8-1255-443c-89f1-5960308f835d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:27.248427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1431d8-1255-443c-89f1-5960308f835d service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:27.266094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 826/1634] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:27 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:27.362404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93006-90bb-48b9-935e-ce8c1c47809b None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:27.362538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93006-90bb-48b9-935e-ce8c1c47809b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:27.362725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93006-90bb-48b9-935e-ce8c1c47809b None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:27.364429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 827/1635] 10.222.0.52 () {54 vars in 912 bytes} [Tue Jun 21 15:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:53:27.370219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab522c3-397f-45e5-ab18-91da5dc82396 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:27.370528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab522c3-397f-45e5-ab18-91da5dc82396 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:27.370819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab522c3-397f-45e5-ab18-91da5dc82396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:27.384999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c85690-8d7c-4d07-b681-740f6d01e306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:27.387865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c85690-8d7c-4d07-b681-740f6d01e306 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:27.404126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-eab522c3-397f-45e5-ab18-91da5dc82396 None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:27.405010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 809/1636] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:27 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:53:27.417919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-eab522c3-397f-45e5-ab18-91da5dc82396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:27.541288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c85690-8d7c-4d07-b681-740f6d01e306 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:27.569752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:27.582194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 828/1637] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:53:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:53:27.600570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab522c3-397f-45e5-ab18-91da5dc82396 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:27.624164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab522c3-397f-45e5-ab18-91da5dc82396 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:27.631804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99439fb-49c6-4cd3-b945-39c0e96eb72d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:27.634112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99439fb-49c6-4cd3-b945-39c0e96eb72d service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:27.636670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99439fb-49c6-4cd3-b945-39c0e96eb72d service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:27.637734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99439fb-49c6-4cd3-b945-39c0e96eb72d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:27.674089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99439fb-49c6-4cd3-b945-39c0e96eb72d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:27.676220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99439fb-49c6-4cd3-b945-39c0e96eb72d service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:27.702227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c85690-8d7c-4d07-b681-740f6d01e306 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:27.712970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:27.717877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 829/1638] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:53:27.722803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:27.723154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:27.723442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] PATH_INFO: `/v3/users/22b55c4684194f2cb19cf31edd9e7adc` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:27.725179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b55c4684194f2cb19cf31edd9e7adc)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:27.727622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:27.824467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b55c4684194f2cb19cf31edd9e7adc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:27.869958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b55c4684194f2cb19cf31edd9e7adc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:27.889194 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b55c4684194f2cb19cf31edd9e7adc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:27.911400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b55c4684194f2cb19cf31edd9e7adc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:27.912174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b55c4684194f2cb19cf31edd9e7adc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:27.933209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 810/1639] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:27 2022] DELETE /identity/v3/users/22b55c4684194f2cb19cf31edd9e7adc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:27.933724 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee56b99-02c2-4b54-bffe-4f888cd264e6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:27.934090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee56b99-02c2-4b54-bffe-4f888cd264e6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:27.935067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee56b99-02c2-4b54-bffe-4f888cd264e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:28.001691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-8ee56b99-02c2-4b54-bffe-4f888cd264e6 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:28.004968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8ee56b99-02c2-4b54-bffe-4f888cd264e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:28.200625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 830/1640] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:28.224518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:28.226452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:28.254301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee56b99-02c2-4b54-bffe-4f888cd264e6 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:28.256227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8ee56b99-02c2-4b54-bffe-4f888cd264e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:28.277718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee56b99-02c2-4b54-bffe-4f888cd264e6 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:28.287304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:28.288551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:28.288844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:28.289890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] PATH_INFO: `/v3/users/f92d8614e59b45dfa6d264a46173690a` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:28.293038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92d8614e59b45dfa6d264a46173690a)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:28.294141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: WARNING py.warnings [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:53:28.294141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: warnings.warn(msg) Jun 21 15:53:28.294141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: Jun 21 15:53:28.295570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:28.354684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe258bc-1c8e-4f9c-8677-a14f8f12b641 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:28.359418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd86f7-6c1f-4ad6-9a04-9935b05307f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:28.360088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd86f7-6c1f-4ad6-9a04-9935b05307f6 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:28.360485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd86f7-6c1f-4ad6-9a04-9935b05307f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:28.360698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd86f7-6c1f-4ad6-9a04-9935b05307f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:28.373135 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd86f7-6c1f-4ad6-9a04-9935b05307f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:28.375124 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd86f7-6c1f-4ad6-9a04-9935b05307f6 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:28.385659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:28.395697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 811/1641] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:28.440768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92d8614e59b45dfa6d264a46173690a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:28.491332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92d8614e59b45dfa6d264a46173690a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:28.503673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92d8614e59b45dfa6d264a46173690a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:28.505404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92d8614e59b45dfa6d264a46173690a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:28.527230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92d8614e59b45dfa6d264a46173690a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:28.535563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 831/1642] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:28 2022] DELETE /identity/v3/users/f92d8614e59b45dfa6d264a46173690a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:28.542293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95788791-a10e-489e-aa93-1e5515ec22f0 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:28.544008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95788791-a10e-489e-aa93-1e5515ec22f0 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:28.544840 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95788791-a10e-489e-aa93-1e5515ec22f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:28.605477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-95788791-a10e-489e-aa93-1e5515ec22f0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:28.609151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-95788791-a10e-489e-aa93-1e5515ec22f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:28.699939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:28.798303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 812/1643] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:28.861998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:28.864365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:28.996300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd9db3-fb53-4423-9d7f-748bc0ca1654 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:29.009758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77735128-41ac-4f3f-acdf-99111bc4a1cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:29.010520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77735128-41ac-4f3f-acdf-99111bc4a1cb service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:29.010807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77735128-41ac-4f3f-acdf-99111bc4a1cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:29.011092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77735128-41ac-4f3f-acdf-99111bc4a1cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:29.032220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77735128-41ac-4f3f-acdf-99111bc4a1cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:29.033741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77735128-41ac-4f3f-acdf-99111bc4a1cb service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:29.061829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 832/1644] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:29.334352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70316f5-d1d1-4921-9e08-9c85562cd5ab None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:29.334542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70316f5-d1d1-4921-9e08-9c85562cd5ab None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:29.335197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70316f5-d1d1-4921-9e08-9c85562cd5ab None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:29.338356 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 813/1645] 10.222.0.52 () {54 vars in 961 bytes} [Tue Jun 21 15:53:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:53:29.353627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a8498-a17e-496b-92d8-19a77d3b9f99 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:29.354476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a8498-a17e-496b-92d8-19a77d3b9f99 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:29.354774 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a8498-a17e-496b-92d8-19a77d3b9f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:29.388127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-283a8498-a17e-496b-92d8-19a77d3b9f99 None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:29.392884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-283a8498-a17e-496b-92d8-19a77d3b9f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:29.590339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 833/1646] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Jun 21 15:53:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:29.591713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1139a10-2753-43fc-9d66-378aa3bcbe75 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:29.592477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1139a10-2753-43fc-9d66-378aa3bcbe75 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:29.592477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1139a10-2753-43fc-9d66-378aa3bcbe75 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:29.593974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 814/1647] 10.222.0.52 () {54 vars in 912 bytes} [Tue Jun 21 15:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:53:29.600828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3267c-2131-4093-a1b3-c5039ee22bba None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:29.601658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3267c-2131-4093-a1b3-c5039ee22bba None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:29.602210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3267c-2131-4093-a1b3-c5039ee22bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:29.636807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-09c3267c-2131-4093-a1b3-c5039ee22bba None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:29.640104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-09c3267c-2131-4093-a1b3-c5039ee22bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:29.655138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 834/1648] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:53:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:29.668322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1139a10-2753-43fc-9d66-378aa3bcbe75 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:29.670673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f1139a10-2753-43fc-9d66-378aa3bcbe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:29.682619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a48e1f-83a1-4e7a-8679-7f8c4665bed2 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:29.682971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a48e1f-83a1-4e7a-8679-7f8c4665bed2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:29.683177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a48e1f-83a1-4e7a-8679-7f8c4665bed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:29.743665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-33a48e1f-83a1-4e7a-8679-7f8c4665bed2 None None] MFA Rules not processed for user `76f88f8b791743e3a6b7cb0d29f3b4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:29.745228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-33a48e1f-83a1-4e7a-8679-7f8c4665bed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:29.769978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:29.778634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1139a10-2753-43fc-9d66-378aa3bcbe75 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:29.783377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:29.786929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:29.791844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:29.792248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:29.816886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:29.818832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:29.861414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 815/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:29.906405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 835/1650] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:29.910945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:29.912407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:29.935957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab0f94-9e1e-4621-9083-c554aafc7b2a service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:29.955235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:29.956245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:29.956649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:29.957002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:29.957831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:29.959908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:29.997021 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 816/1651] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:29 2022] POST /identity/v3/projects => generated 405 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:29.999183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a48e1f-83a1-4e7a-8679-7f8c4665bed2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:30.054070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:30.057809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:30.077152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a48e1f-83a1-4e7a-8679-7f8c4665bed2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:30.226296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded382df-63d2-492e-99c8-aa9e0a23cdf2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:30.246493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:30.248810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:30.249195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:30.249685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] PATH_INFO: `/v3/projects/3be3a5d422414cb490c344af063afc60` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:30.252185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be3a5d422414cb490c344af063afc60)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:30.253267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: WARNING py.warnings [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 21 15:53:30.253267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: warnings.warn(msg) Jun 21 15:53:30.253267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: Jun 21 15:53:30.254757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:30.284632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb96510-7c6d-4916-a208-1fbf50ab53d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:30.285272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb96510-7c6d-4916-a208-1fbf50ab53d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:30.285552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb96510-7c6d-4916-a208-1fbf50ab53d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:30.285845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb96510-7c6d-4916-a208-1fbf50ab53d9 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:30.286558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb96510-7c6d-4916-a208-1fbf50ab53d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:30.288949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb96510-7c6d-4916-a208-1fbf50ab53d9 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:30.332441 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 836/1652] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:29 2022] POST /identity/v3/users => generated 309 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:30.338110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb96510-7c6d-4916-a208-1fbf50ab53d9 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:30.361884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb96510-7c6d-4916-a208-1fbf50ab53d9 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:30.391321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be3a5d422414cb490c344af063afc60', 'request_id': 'req-34679da4-3710-4817-a6be-373da1245e87'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.434935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be3a5d422414cb490c344af063afc60', 'request_id': 'req-34679da4-3710-4817-a6be-373da1245e87'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.441212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:30.446153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be3a5d422414cb490c344af063afc60', 'request_id': 'req-34679da4-3710-4817-a6be-373da1245e87'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.489211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] The token cache is being invalidate because project 3be3a5d422414cb490c344af063afc60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:30.489211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.493964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.498946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 817/1653] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:30 2022] DELETE /identity/v3/projects/3be3a5d422414cb490c344af063afc60 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:30.502358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:30.504016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:30.506783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:30.507931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:30.508439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:30.508869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:30.509767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:30.511578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:30.520502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34679da4-3710-4817-a6be-373da1245e87 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:30.538138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:30.539754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:30.540217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:30.540663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] PATH_INFO: `/v3/users/abd5faecbc39415c884a8412f7a26c84` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:30.542751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd5faecbc39415c884a8412f7a26c84)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:30.543668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 837/1654] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:30.544593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:30.546383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:30.547026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:30.595161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a4e8d-43a5-4c99-a941-cd0bc225ba4c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:30.613313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c07e7-73b9-47ef-a0f7-889d1bb5d679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:30.614347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c07e7-73b9-47ef-a0f7-889d1bb5d679 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:30.614631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c07e7-73b9-47ef-a0f7-889d1bb5d679 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:30.614910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c07e7-73b9-47ef-a0f7-889d1bb5d679 admin admin] PATH_INFO: `/v3/projects/e7ec32cfb4b843309d5983886dd6fb8d/users/31219f8bb890473588d968c33b170e6f/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:30.651130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c07e7-73b9-47ef-a0f7-889d1bb5d679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ec32cfb4b843309d5983886dd6fb8d, user_id=31219f8bb890473588d968c33b170e6f, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:30.657288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c07e7-73b9-47ef-a0f7-889d1bb5d679 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:30.685513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 838/1655] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:30 2022] PUT /identity/v3/projects/e7ec32cfb4b843309d5983886dd6fb8d/users/31219f8bb890473588d968c33b170e6f/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:30.689938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c07e7-73b9-47ef-a0f7-889d1bb5d679 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:30.789113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd5faecbc39415c884a8412f7a26c84'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.789113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd5faecbc39415c884a8412f7a26c84'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.789113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd5faecbc39415c884a8412f7a26c84'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.789113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd5faecbc39415c884a8412f7a26c84'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.789113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd5faecbc39415c884a8412f7a26c84'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:30.791534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:30.796212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 818/1656] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:30 2022] DELETE /identity/v3/users/abd5faecbc39415c884a8412f7a26c84 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:53:30.872769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c07e7-73b9-47ef-a0f7-889d1bb5d679 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:31.019006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:31.020923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:31.023783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:31.024093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:31.026087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:31.030912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:31.081381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 839/1657] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:31.085800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:31.087317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:31.107380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b6006-cfc5-4ab5-8659-25f403df9e42 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:31.113405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:31.115155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:31.115663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:31.116214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] PATH_INFO: `/v3/projects/e7ec32cfb4b843309d5983886dd6fb8d/users/31219f8bb890473588d968c33b170e6f/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:31.119434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ec32cfb4b843309d5983886dd6fb8d, user_id=31219f8bb890473588d968c33b170e6f, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:31.122557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:31.147476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 819/1658] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:31 2022] PUT /identity/v3/projects/e7ec32cfb4b843309d5983886dd6fb8d/users/31219f8bb890473588d968c33b170e6f/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:31.386898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:31.388969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:31.422363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78faced-e6aa-41b0-bcf5-2d68763f525d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:31.633958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:31.635579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:31.636022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:31.636309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] PATH_INFO: `/v3/projects/2281fb3898054e0881b900586871d5ab` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:31.644020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2281fb3898054e0881b900586871d5ab)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:31.646214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:31.762181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2281fb3898054e0881b900586871d5ab', 'request_id': 'req-f3caca21-0664-436d-8986-378b24f001cd'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:31.786508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2281fb3898054e0881b900586871d5ab', 'request_id': 'req-f3caca21-0664-436d-8986-378b24f001cd'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:31.827832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2281fb3898054e0881b900586871d5ab', 'request_id': 'req-f3caca21-0664-436d-8986-378b24f001cd'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:31.827832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:31.840469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] The token cache is being invalidate because project 2281fb3898054e0881b900586871d5ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:31.840469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:31.858785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:31.874200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 840/1659] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:31 2022] DELETE /identity/v3/projects/2281fb3898054e0881b900586871d5ab => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:31.874364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:31.876423 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:31.915771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77afe2-202a-42ed-85cd-aedf5e25e1ef admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:32.059909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:32.060813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:32.061094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:32.061524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] PATH_INFO: `/v3/users/212f41d7d26b4ed1810c6672dfa360db` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:32.063341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212f41d7d26b4ed1810c6672dfa360db)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:32.065625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:32.139944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:32.144152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212f41d7d26b4ed1810c6672dfa360db'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:32.172650 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212f41d7d26b4ed1810c6672dfa360db'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:32.187271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212f41d7d26b4ed1810c6672dfa360db'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:32.200987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212f41d7d26b4ed1810c6672dfa360db'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:32.201955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fdf30e66-858f-4a57-8962-9cc88e8263b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212f41d7d26b4ed1810c6672dfa360db'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:32.215535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 820/1660] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:31 2022] DELETE /identity/v3/users/212f41d7d26b4ed1810c6672dfa360db => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:32.885974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:32.888066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:32.914251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caca21-0664-436d-8986-378b24f001cd admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:33.077475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:33.090831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:33.091469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:33.091976 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] PATH_INFO: `/v3/projects/97f65c3678af4f3eba2297ba89936387` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:33.094215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f65c3678af4f3eba2297ba89936387)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:33.100742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:33.195212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f65c3678af4f3eba2297ba89936387', 'request_id': 'req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:33.222560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f65c3678af4f3eba2297ba89936387', 'request_id': 'req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:33.240674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f65c3678af4f3eba2297ba89936387', 'request_id': 'req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:33.263770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] The token cache is being invalidate because project 97f65c3678af4f3eba2297ba89936387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:33.264062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:33.266712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1ef7bc59-fd2d-4499-ba25-867a9f0d59b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:33.273474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 841/1661] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:32 2022] DELETE /identity/v3/projects/97f65c3678af4f3eba2297ba89936387 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:33.275096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfff7a7-7877-49e9-b55e-2f944d0c2298 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:33.275540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfff7a7-7877-49e9-b55e-2f944d0c2298 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:33.275705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfff7a7-7877-49e9-b55e-2f944d0c2298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:33.308788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0dfff7a7-7877-49e9-b55e-2f944d0c2298 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:33.310581 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0dfff7a7-7877-49e9-b55e-2f944d0c2298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:33.478322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 821/1662] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:33.487163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053747-122c-44b7-b904-d3123fd1766d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:33.487741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053747-122c-44b7-b904-d3123fd1766d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:33.488227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17053747-122c-44b7-b904-d3123fd1766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:33.524715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-17053747-122c-44b7-b904-d3123fd1766d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:33.526005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-17053747-122c-44b7-b904-d3123fd1766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:33.544232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 842/1663] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:33.548740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfff7a7-7877-49e9-b55e-2f944d0c2298 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:33.562100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfff7a7-7877-49e9-b55e-2f944d0c2298 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:33.585144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:33.586027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:33.587520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:33.587807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:33.588596 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:33.588909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:33.596142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:33.635075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 822/1664] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:33.639207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053747-122c-44b7-b904-d3123fd1766d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:33.651863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17053747-122c-44b7-b904-d3123fd1766d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:33.666498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c120e00-2666-49ed-962e-5283f628216b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:33.667234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120e00-2666-49ed-962e-5283f628216b admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:33.667560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120e00-2666-49ed-962e-5283f628216b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:33.667840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c120e00-2666-49ed-962e-5283f628216b admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:33.668897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c120e00-2666-49ed-962e-5283f628216b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:33.674119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c120e00-2666-49ed-962e-5283f628216b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:33.736764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 843/1665] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:33 2022] POST /identity/v3/projects => generated 357 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:33.745243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:33.784938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac26227-2cd5-41bc-9ee8-7d44d5e11f79 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:33.949706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117e4c6-b350-4f2f-af71-f5a46b97e411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:33.951080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e4c6-b350-4f2f-af71-f5a46b97e411 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:33.951729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e4c6-b350-4f2f-af71-f5a46b97e411 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:33.952278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e4c6-b350-4f2f-af71-f5a46b97e411 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:33.953301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117e4c6-b350-4f2f-af71-f5a46b97e411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:33.957330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117e4c6-b350-4f2f-af71-f5a46b97e411 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:34.019745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 823/1666] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:33 2022] POST /identity/v3/users => generated 279 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:53:34.020770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c120e00-2666-49ed-962e-5283f628216b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:34.060649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c120e00-2666-49ed-962e-5283f628216b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:34.073765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa192723-bba3-4858-872c-6abeb67de845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:34.074839 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa192723-bba3-4858-872c-6abeb67de845 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.075303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa192723-bba3-4858-872c-6abeb67de845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.076019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa192723-bba3-4858-872c-6abeb67de845 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.077996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa192723-bba3-4858-872c-6abeb67de845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:34.083027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa192723-bba3-4858-872c-6abeb67de845 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:34.108821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 844/1667] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:53:34.114028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117e4c6-b350-4f2f-af71-f5a46b97e411 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:34.131493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117e4c6-b350-4f2f-af71-f5a46b97e411 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:34.139738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:34.141023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.141376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.141748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.143430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:34.145508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:34.169425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 824/1668] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:34.170581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa192723-bba3-4858-872c-6abeb67de845 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:34.185544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa192723-bba3-4858-872c-6abeb67de845 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:34.202879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f87bfe-91e8-4615-b522-b55e6c3aaabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:34.203961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f87bfe-91e8-4615-b522-b55e6c3aaabe admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.204265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f87bfe-91e8-4615-b522-b55e6c3aaabe admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.204534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f87bfe-91e8-4615-b522-b55e6c3aaabe admin admin] PATH_INFO: `/v3/projects/5dd4fcc787e844d9997c1ae283584e09/users/dd3b26c356e34584a38eceea2ec1f57f/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.210501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:34.211993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:34.242726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f87bfe-91e8-4615-b522-b55e6c3aaabe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd4fcc787e844d9997c1ae283584e09, user_id=dd3b26c356e34584a38eceea2ec1f57f, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:34.245617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f87bfe-91e8-4615-b522-b55e6c3aaabe admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:34.260152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73757c6b-61eb-4177-831c-e53b199ee171 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:34.273195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:34.275678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.276323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.278380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] PATH_INFO: `/v3/projects/1675afbdcaec408e8d7049a1932c9690` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.280237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 845/1669] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:34 2022] PUT /identity/v3/projects/5dd4fcc787e844d9997c1ae283584e09/users/dd3b26c356e34584a38eceea2ec1f57f/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:53:34.290728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1675afbdcaec408e8d7049a1932c9690)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:34.293074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:34.293800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b623-f657-4584-89ab-51ecfc9a3ac4 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.294087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b623-f657-4584-89ab-51ecfc9a3ac4 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.294420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b623-f657-4584-89ab-51ecfc9a3ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.356769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5c89b623-f657-4584-89ab-51ecfc9a3ac4 None None] MFA Rules not processed for user `dd3b26c356e34584a38eceea2ec1f57f`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:34.358693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5c89b623-f657-4584-89ab-51ecfc9a3ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:34.366959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:34.379644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:34.469743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1675afbdcaec408e8d7049a1932c9690', 'request_id': 'req-24bd342f-f511-4809-a2d1-a18478665c53'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:34.486603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1675afbdcaec408e8d7049a1932c9690', 'request_id': 'req-24bd342f-f511-4809-a2d1-a18478665c53'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:34.501959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1675afbdcaec408e8d7049a1932c9690', 'request_id': 'req-24bd342f-f511-4809-a2d1-a18478665c53'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:34.532273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] The token cache is being invalidate because project 1675afbdcaec408e8d7049a1932c9690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:34.534467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:34.535706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24bd342f-f511-4809-a2d1-a18478665c53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:34.537860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 825/1670] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:34 2022] DELETE /identity/v3/projects/1675afbdcaec408e8d7049a1932c9690 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:34.544193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487463-05e0-4d5f-afac-93d03b260c1b None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.544822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487463-05e0-4d5f-afac-93d03b260c1b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.545601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df487463-05e0-4d5f-afac-93d03b260c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.557596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 846/1671] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:34 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:34.561841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89b623-f657-4584-89ab-51ecfc9a3ac4 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:34.586498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-df487463-05e0-4d5f-afac-93d03b260c1b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:34.588188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-df487463-05e0-4d5f-afac-93d03b260c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:34.735310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89b623-f657-4584-89ab-51ecfc9a3ac4 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:34.850321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 826/1672] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:34.858139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e1f1-f027-47c0-99a7-e998ecfd9ceb None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.858913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e1f1-f027-47c0-99a7-e998ecfd9ceb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.859389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12e1f1-f027-47c0-99a7-e998ecfd9ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.903025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd5420-3376-42af-8ab6-638121f9f35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:34.903888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd5420-3376-42af-8ab6-638121f9f35c admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.904323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd5420-3376-42af-8ab6-638121f9f35c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.904797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd5420-3376-42af-8ab6-638121f9f35c admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.905893 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd5420-3376-42af-8ab6-638121f9f35c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:34.908985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd5420-3376-42af-8ab6-638121f9f35c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:34.922929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ef12e1f1-f027-47c0-99a7-e998ecfd9ceb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:34.927343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ef12e1f1-f027-47c0-99a7-e998ecfd9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:34.961813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 827/1673] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:34.963812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0061ec4-ad12-4eb1-b3db-f59ec62ac8cc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:34.964551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0061ec4-ad12-4eb1-b3db-f59ec62ac8cc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:34.965284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0061ec4-ad12-4eb1-b3db-f59ec62ac8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:34.988159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 847/1674] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:34 2022] POST /identity/v3/projects => generated 355 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:53:34.991519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd5420-3376-42af-8ab6-638121f9f35c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:35.091242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b0061ec4-ad12-4eb1-b3db-f59ec62ac8cc None None] MFA Rules not processed for user `31219f8bb890473588d968c33b170e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:35.092931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b0061ec4-ad12-4eb1-b3db-f59ec62ac8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:35.153284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:35.225872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd5420-3376-42af-8ab6-638121f9f35c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:35.257538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:35.259938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:35.259938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:35.259938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:35.260360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:35.260533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:35.263110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:35.294002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 848/1675] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:35.300238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:35.304149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:35.335451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185919b-f55d-457c-bec5-b28412f0d6ae admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:35.352897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 828/1676] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:34 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:35.357745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0061ec4-ad12-4eb1-b3db-f59ec62ac8cc None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:35.359855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ba6cf-a57c-4526-a921-315fd0b4a9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:35.360880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ba6cf-a57c-4526-a921-315fd0b4a9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:35.361167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ba6cf-a57c-4526-a921-315fd0b4a9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:35.377958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ba6cf-a57c-4526-a921-315fd0b4a9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:35.378081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ba6cf-a57c-4526-a921-315fd0b4a9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:35.380206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ba6cf-a57c-4526-a921-315fd0b4a9c9 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:35.422164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0061ec4-ad12-4eb1-b3db-f59ec62ac8cc None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:35.425169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 849/1677] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:35 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:35.428861 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ba6cf-a57c-4526-a921-315fd0b4a9c9 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:35.443869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a826659-0d2e-41a1-9c23-52a1cc9d9dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:35.444980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a826659-0d2e-41a1-9c23-52a1cc9d9dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:35.445493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a826659-0d2e-41a1-9c23-52a1cc9d9dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:35.445811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a826659-0d2e-41a1-9c23-52a1cc9d9dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:35.446546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a826659-0d2e-41a1-9c23-52a1cc9d9dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:35.448596 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a826659-0d2e-41a1-9c23-52a1cc9d9dc2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:35.466331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ba6cf-a57c-4526-a921-315fd0b4a9c9 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:35.487044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9235a-edd7-4a14-aa13-b1f09aac0e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:35.487756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9235a-edd7-4a14-aa13-b1f09aac0e8d admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:35.488217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9235a-edd7-4a14-aa13-b1f09aac0e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:35.488584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9235a-edd7-4a14-aa13-b1f09aac0e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:35.489926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9235a-edd7-4a14-aa13-b1f09aac0e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:35.491698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9235a-edd7-4a14-aa13-b1f09aac0e8d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:35.548367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 850/1678] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:53:35.550305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9235a-edd7-4a14-aa13-b1f09aac0e8d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:35.561056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 829/1679] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:35 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:35.566558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a826659-0d2e-41a1-9c23-52a1cc9d9dc2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:35.586418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9235a-edd7-4a14-aa13-b1f09aac0e8d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:35.591292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:35.739802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a826659-0d2e-41a1-9c23-52a1cc9d9dc2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:35.756387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e94bdf-62c2-46bf-aba2-95d3ee1a96f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:35.757300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e94bdf-62c2-46bf-aba2-95d3ee1a96f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:35.757632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e94bdf-62c2-46bf-aba2-95d3ee1a96f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:35.758014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e94bdf-62c2-46bf-aba2-95d3ee1a96f9 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:35.758783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e94bdf-62c2-46bf-aba2-95d3ee1a96f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:35.761279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e94bdf-62c2-46bf-aba2-95d3ee1a96f9 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:35.789945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd891d57-4971-4d4d-b458-f563ba5cd7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:35.794298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891d57-4971-4d4d-b458-f563ba5cd7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:35.794540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891d57-4971-4d4d-b458-f563ba5cd7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:35.794824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891d57-4971-4d4d-b458-f563ba5cd7f1 admin admin] PATH_INFO: `/v3/projects/92782533b4c44f1bbc7b425a768ab6e6/users/4a653b37ea3a4270816a9993bd2e94d3/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:35.835334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 830/1680] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:35 2022] POST /identity/v3/users => generated 291 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:53:35.838105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e94bdf-62c2-46bf-aba2-95d3ee1a96f9 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:35.868413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd891d57-4971-4d4d-b458-f563ba5cd7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92782533b4c44f1bbc7b425a768ab6e6, user_id=4a653b37ea3a4270816a9993bd2e94d3, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:35.877645 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd891d57-4971-4d4d-b458-f563ba5cd7f1 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:35.892000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e94bdf-62c2-46bf-aba2-95d3ee1a96f9 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:35.915083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e818b-8cf1-4bc7-b012-9f5331aa3250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:35.915083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e818b-8cf1-4bc7-b012-9f5331aa3250 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:35.915083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e818b-8cf1-4bc7-b012-9f5331aa3250 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:35.915083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e818b-8cf1-4bc7-b012-9f5331aa3250 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:35.922829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e818b-8cf1-4bc7-b012-9f5331aa3250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:35.926249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 851/1681] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:35 2022] PUT /identity/v3/projects/92782533b4c44f1bbc7b425a768ab6e6/users/4a653b37ea3a4270816a9993bd2e94d3/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:35.928858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e818b-8cf1-4bc7-b012-9f5331aa3250 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:35.933292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd891d57-4971-4d4d-b458-f563ba5cd7f1 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:35.965962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 831/1682] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:35.966987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e818b-8cf1-4bc7-b012-9f5331aa3250 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:36.014316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e818b-8cf1-4bc7-b012-9f5331aa3250 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:36.204813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd891d57-4971-4d4d-b458-f563ba5cd7f1 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:36.232337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1ab91-0b90-48d8-b8d1-9aad8150d9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:36.234462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1ab91-0b90-48d8-b8d1-9aad8150d9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:36.234748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1ab91-0b90-48d8-b8d1-9aad8150d9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:36.237186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1ab91-0b90-48d8-b8d1-9aad8150d9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:36.240407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1ab91-0b90-48d8-b8d1-9aad8150d9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:36.252021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1ab91-0b90-48d8-b8d1-9aad8150d9a2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:36.286639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24582d-f613-434a-9172-9340ca7d6b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:36.287884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24582d-f613-434a-9172-9340ca7d6b55 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:36.292984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24582d-f613-434a-9172-9340ca7d6b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:36.295911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24582d-f613-434a-9172-9340ca7d6b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:36.296965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24582d-f613-434a-9172-9340ca7d6b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:36.299095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24582d-f613-434a-9172-9340ca7d6b55 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:36.306786 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 852/1683] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:36.311110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1ab91-0b90-48d8-b8d1-9aad8150d9a2 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:36.339468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 832/1684] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:36.341621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1ab91-0b90-48d8-b8d1-9aad8150d9a2 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:36.344033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24582d-f613-434a-9172-9340ca7d6b55 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:36.352600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb217dd2-d9b2-457f-b2f4-f5e9aa2dbfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:36.353667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb217dd2-d9b2-457f-b2f4-f5e9aa2dbfea admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:36.354015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb217dd2-d9b2-457f-b2f4-f5e9aa2dbfea admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:36.354354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb217dd2-d9b2-457f-b2f4-f5e9aa2dbfea admin admin] PATH_INFO: `/v3/projects/92782533b4c44f1bbc7b425a768ab6e6/users/4a653b37ea3a4270816a9993bd2e94d3/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:36.356872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb217dd2-d9b2-457f-b2f4-f5e9aa2dbfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=92782533b4c44f1bbc7b425a768ab6e6, user_id=4a653b37ea3a4270816a9993bd2e94d3, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:36.361312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb217dd2-d9b2-457f-b2f4-f5e9aa2dbfea admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:36.367216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24582d-f613-434a-9172-9340ca7d6b55 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:36.393511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a8a12-7cb1-445e-9761-cb5f264e70fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:36.394854 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8a12-7cb1-445e-9761-cb5f264e70fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:36.395378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8a12-7cb1-445e-9761-cb5f264e70fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:36.395474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 853/1685] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:36 2022] PUT /identity/v3/projects/92782533b4c44f1bbc7b425a768ab6e6/users/4a653b37ea3a4270816a9993bd2e94d3/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:36.395935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8a12-7cb1-445e-9761-cb5f264e70fd admin admin] PATH_INFO: `/v3/projects/b9bf6d48896d42a3b45ff20a9790f79b/users/aabdbb0cb065463bb778db01ce2f004a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:36.402735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430033ad-65db-408c-9488-7ae32839fba5 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:36.403011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430033ad-65db-408c-9488-7ae32839fba5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:36.403225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430033ad-65db-408c-9488-7ae32839fba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:36.430071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a8a12-7cb1-445e-9761-cb5f264e70fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bf6d48896d42a3b45ff20a9790f79b, user_id=aabdbb0cb065463bb778db01ce2f004a, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:36.434050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a8a12-7cb1-445e-9761-cb5f264e70fd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:36.453398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:36.461231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 833/1686] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:36 2022] PUT /identity/v3/projects/b9bf6d48896d42a3b45ff20a9790f79b/users/aabdbb0cb065463bb778db01ce2f004a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:36.463661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-430033ad-65db-408c-9488-7ae32839fba5 None None] MFA Rules not processed for user `4a653b37ea3a4270816a9993bd2e94d3`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:36.465711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-430033ad-65db-408c-9488-7ae32839fba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:36.471919 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15078af-0be6-41ab-a71f-7b4ae4b83bb9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:36.472458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15078af-0be6-41ab-a71f-7b4ae4b83bb9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:36.472980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15078af-0be6-41ab-a71f-7b4ae4b83bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:36.538352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d15078af-0be6-41ab-a71f-7b4ae4b83bb9 None None] MFA Rules not processed for user `aabdbb0cb065463bb778db01ce2f004a`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:36.541258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d15078af-0be6-41ab-a71f-7b4ae4b83bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:36.672991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 854/1687] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:36 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:36.681405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:36.682971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430033ad-65db-408c-9488-7ae32839fba5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:36.686243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-430033ad-65db-408c-9488-7ae32839fba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:36.728906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 834/1688] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:53:36.736157 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15078af-0be6-41ab-a71f-7b4ae4b83bb9 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:36.864688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15078af-0be6-41ab-a71f-7b4ae4b83bb9 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:36.867237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430033ad-65db-408c-9488-7ae32839fba5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:36.876546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc44dca-779e-430e-b7c9-08bae70297d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:36.877241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc44dca-779e-430e-b7c9-08bae70297d8 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:36.877523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc44dca-779e-430e-b7c9-08bae70297d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:36.877815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc44dca-779e-430e-b7c9-08bae70297d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:36.880570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e0f9b-628b-4d13-8d36-c0ef4fb21fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:36.881392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e0f9b-628b-4d13-8d36-c0ef4fb21fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:36.881847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e0f9b-628b-4d13-8d36-c0ef4fb21fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:36.882221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e0f9b-628b-4d13-8d36-c0ef4fb21fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:36.883008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e0f9b-628b-4d13-8d36-c0ef4fb21fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:36.885044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e0f9b-628b-4d13-8d36-c0ef4fb21fe3 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:36.903143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc44dca-779e-430e-b7c9-08bae70297d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:36.904542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc44dca-779e-430e-b7c9-08bae70297d8 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:36.937513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 855/1689] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:36 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:36.940526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 835/1690] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:36 2022] POST /identity/v3/projects => generated 379 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:36.950487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc44dca-779e-430e-b7c9-08bae70297d8 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:36.986080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc44dca-779e-430e-b7c9-08bae70297d8 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:37.120706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e0f9b-628b-4d13-8d36-c0ef4fb21fe3 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:37.217554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e8917-adb5-4a22-bc13-2b8ff7b99347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:37.218189 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e8917-adb5-4a22-bc13-2b8ff7b99347 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:37.218680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e8917-adb5-4a22-bc13-2b8ff7b99347 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:37.219006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e8917-adb5-4a22-bc13-2b8ff7b99347 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:37.219940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e8917-adb5-4a22-bc13-2b8ff7b99347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:37.222006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e8917-adb5-4a22-bc13-2b8ff7b99347 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:37.268871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e0f9b-628b-4d13-8d36-c0ef4fb21fe3 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:37.277171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:37.278832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:37.279228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:37.279974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:37.285757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:37.309030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 856/1691] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:36 2022] POST /identity/v3/users => generated 296 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:37.310259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e8917-adb5-4a22-bc13-2b8ff7b99347 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:37.339842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e8917-adb5-4a22-bc13-2b8ff7b99347 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:37.355638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca225e5-85fb-442d-a8f5-f9398c3a03e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:37.356335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca225e5-85fb-442d-a8f5-f9398c3a03e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:37.356613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca225e5-85fb-442d-a8f5-f9398c3a03e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:37.356882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca225e5-85fb-442d-a8f5-f9398c3a03e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:37.357620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca225e5-85fb-442d-a8f5-f9398c3a03e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:37.360706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca225e5-85fb-442d-a8f5-f9398c3a03e8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:37.396876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 857/1692] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:37.405445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca225e5-85fb-442d-a8f5-f9398c3a03e8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:37.438597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca225e5-85fb-442d-a8f5-f9398c3a03e8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:37.467227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062c95e-ecb7-4b7f-adc3-4556af098cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:37.468341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062c95e-ecb7-4b7f-adc3-4556af098cfc admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:37.468583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062c95e-ecb7-4b7f-adc3-4556af098cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:37.468963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062c95e-ecb7-4b7f-adc3-4556af098cfc admin admin] PATH_INFO: `/v3/projects/741114fb60f5466f85ee8f4beefc7ab2/users/68fe9655abf34c27945519cdffc89a6a/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:37.511649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062c95e-ecb7-4b7f-adc3-4556af098cfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=741114fb60f5466f85ee8f4beefc7ab2, user_id=68fe9655abf34c27945519cdffc89a6a, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:37.520201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:37.521586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:37.527990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062c95e-ecb7-4b7f-adc3-4556af098cfc admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:37.534061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:37.534744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 836/1693] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:37 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:53:37.576263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 858/1694] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:37 2022] PUT /identity/v3/projects/741114fb60f5466f85ee8f4beefc7ab2/users/68fe9655abf34c27945519cdffc89a6a/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:37.580949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:37.611172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b239c54-7829-48ac-b857-292f6aa588b2 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:37.718521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:37.810723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbcb43-4ce6-4951-8204-5fb661432421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:37.812778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbcb43-4ce6-4951-8204-5fb661432421 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:37.813079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbcb43-4ce6-4951-8204-5fb661432421 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:37.813366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbcb43-4ce6-4951-8204-5fb661432421 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:37.818345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbcb43-4ce6-4951-8204-5fb661432421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:37.824819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbcb43-4ce6-4951-8204-5fb661432421 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:37.868309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 837/1695] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:37.872327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062c95e-ecb7-4b7f-adc3-4556af098cfc admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:37.895722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062c95e-ecb7-4b7f-adc3-4556af098cfc admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:37.909559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7708f3-2c4c-4399-8e4e-2e1354980e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:37.910712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7708f3-2c4c-4399-8e4e-2e1354980e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:37.911140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7708f3-2c4c-4399-8e4e-2e1354980e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:37.911520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7708f3-2c4c-4399-8e4e-2e1354980e57 admin admin] PATH_INFO: `/v3/projects/741114fb60f5466f85ee8f4beefc7ab2/users/68fe9655abf34c27945519cdffc89a6a/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:37.918194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7708f3-2c4c-4399-8e4e-2e1354980e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=741114fb60f5466f85ee8f4beefc7ab2, user_id=68fe9655abf34c27945519cdffc89a6a, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:37.921071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7708f3-2c4c-4399-8e4e-2e1354980e57 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:37.999193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 859/1696] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:37 2022] PUT /identity/v3/projects/741114fb60f5466f85ee8f4beefc7ab2/users/68fe9655abf34c27945519cdffc89a6a/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:38.002652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7504bf-9589-4a2d-a4b5-383307ff16cb None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:38.003172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7504bf-9589-4a2d-a4b5-383307ff16cb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:38.003638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7504bf-9589-4a2d-a4b5-383307ff16cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:38.050810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8a7504bf-9589-4a2d-a4b5-383307ff16cb None None] MFA Rules not processed for user `68fe9655abf34c27945519cdffc89a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:38.052315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8a7504bf-9589-4a2d-a4b5-383307ff16cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:38.172934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 838/1697] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:38.180411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7708f3-2c4c-4399-8e4e-2e1354980e57 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:38.293886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7708f3-2c4c-4399-8e4e-2e1354980e57 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:38.302450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09795f-3589-4d99-9ed7-f35a1772eace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:38.303305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09795f-3589-4d99-9ed7-f35a1772eace service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:38.303718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09795f-3589-4d99-9ed7-f35a1772eace service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:38.304004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09795f-3589-4d99-9ed7-f35a1772eace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:38.339334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09795f-3589-4d99-9ed7-f35a1772eace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:38.341007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09795f-3589-4d99-9ed7-f35a1772eace service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:38.363658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 860/1698] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:38.969283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a3696-4b60-43ef-993d-2e2430b64a19 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:38.969742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a3696-4b60-43ef-993d-2e2430b64a19 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:38.969838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a3696-4b60-43ef-993d-2e2430b64a19 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:38.971481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 839/1699] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:53:39.014107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09795f-3589-4d99-9ed7-f35a1772eace service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:39.030740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09795f-3589-4d99-9ed7-f35a1772eace service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:39.038534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337affd7-6ce8-49c1-afca-7872c5459d14 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:39.038849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337affd7-6ce8-49c1-afca-7872c5459d14 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:39.039191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337affd7-6ce8-49c1-afca-7872c5459d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:39.086005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68fe9655abf34c27945519cdffc89a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '741114fb60f5466f85ee8f4beefc7ab2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:39.086696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:39.086988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:39.087354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:39.088010 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:39.088358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:39.090098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:39.115049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-337affd7-6ce8-49c1-afca-7872c5459d14 None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:39.119451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-337affd7-6ce8-49c1-afca-7872c5459d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:39.150825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 861/1700] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:39.159778 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:39.202012 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fb1fa-780b-4f0f-942f-0507cd5d07f7 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:39.221086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c4b02-a10a-4699-bf40-72170dd0bc09 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68fe9655abf34c27945519cdffc89a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '741114fb60f5466f85ee8f4beefc7ab2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:39.222405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4b02-a10a-4699-bf40-72170dd0bc09 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:39.223941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4b02-a10a-4699-bf40-72170dd0bc09 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:39.224243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c4b02-a10a-4699-bf40-72170dd0bc09 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:39.227350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c4b02-a10a-4699-bf40-72170dd0bc09 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:39.251401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c4b02-a10a-4699-bf40-72170dd0bc09 tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:39.325496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 840/1701] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:53:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:39.356513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 862/1702] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:39 2022] POST /identity/v3/projects => generated 355 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:39.359425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337affd7-6ce8-49c1-afca-7872c5459d14 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:39.379668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337affd7-6ce8-49c1-afca-7872c5459d14 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:39.531808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:39.567544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68fe9655abf34c27945519cdffc89a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '741114fb60f5466f85ee8f4beefc7ab2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:39.568278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:39.568570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:39.568900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:39.569664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:39.570006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:39.575723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:39.585902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4508a16-8606-4fed-8094-59620be2873a None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:39.586097 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4508a16-8606-4fed-8094-59620be2873a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:39.586286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4508a16-8606-4fed-8094-59620be2873a None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:39.588233 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 863/1703] 10.222.0.52 () {54 vars in 960 bytes} [Tue Jun 21 15:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:53:39.597502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7f515-6064-4360-bb3b-c522909bf39d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:39.597815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7f515-6064-4360-bb3b-c522909bf39d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:39.598280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7f515-6064-4360-bb3b-c522909bf39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:39.632119 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 841/1704] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:39.659279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f2a7f515-6064-4360-bb3b-c522909bf39d None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:39.662931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f2a7f515-6064-4360-bb3b-c522909bf39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:39.754657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:39.772683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf8380-26ff-46b0-99e2-3d79017730cd tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:39.796626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68fe9655abf34c27945519cdffc89a6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '741114fb60f5466f85ee8f4beefc7ab2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:39.803661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:39.804062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:39.804496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] PATH_INFO: `/v3/projects/dad2b1069ddb4aa0a1fd60cd6ed0220c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:39.808185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Authorizing `identity:delete_project(project_id=dad2b1069ddb4aa0a1fd60cd6ed0220c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:39.815160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:39.940602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 864/1705] 10.222.0.52 () {58 vars in 1057 bytes} [Tue Jun 21 15:53:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:53:39.948920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510b25-1ab4-42f4-b1df-259bc741cb24 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:39.949266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510b25-1ab4-42f4-b1df-259bc741cb24 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:39.949548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510b25-1ab4-42f4-b1df-259bc741cb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:40.019379 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad2b1069ddb4aa0a1fd60cd6ed0220c', 'request_id': 'req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:40.058052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b0510b25-1ab4-42f4-b1df-259bc741cb24 None None] MFA Rules not processed for user `fa5a48626c934dc2aa4f24002970b6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:40.060708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b0510b25-1ab4-42f4-b1df-259bc741cb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:40.062688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad2b1069ddb4aa0a1fd60cd6ed0220c', 'request_id': 'req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:40.078040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad2b1069ddb4aa0a1fd60cd6ed0220c', 'request_id': 'req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:40.096083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] The token cache is being invalidate because project dad2b1069ddb4aa0a1fd60cd6ed0220c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:40.098190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:40.101936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:40.113054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 842/1706] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:39 2022] DELETE /identity/v3/projects/dad2b1069ddb4aa0a1fd60cd6ed0220c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:40.113858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:40.121919 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:40.142170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:40.218987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 865/1707] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:40.236516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510b25-1ab4-42f4-b1df-259bc741cb24 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:40.238074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b0510b25-1ab4-42f4-b1df-259bc741cb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:40.309374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ead11-b1f8-4a8b-807c-d9e2b133dc4e tempest-VolumeQuotaClassesTest-230568934 tempest-VolumeQuotaClassesTest-230568934-project-admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:40.325942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:40.326683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:40.326973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:40.327253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:40.333319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:40.363897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:40.368592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:40.402101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 843/1708] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:40.411732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510b25-1ab4-42f4-b1df-259bc741cb24 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:40.418777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36001551-09a4-4573-8277-7f62852c92df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:40.419548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36001551-09a4-4573-8277-7f62852c92df service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:40.420172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36001551-09a4-4573-8277-7f62852c92df service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:40.420729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36001551-09a4-4573-8277-7f62852c92df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:40.432240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36001551-09a4-4573-8277-7f62852c92df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:40.434033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36001551-09a4-4573-8277-7f62852c92df service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:40.472711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 866/1709] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:40.578207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:40.580539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:40.758051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36001551-09a4-4573-8277-7f62852c92df service nova] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:40.761406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-36001551-09a4-4573-8277-7f62852c92df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:40.772383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:40.826679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c099a-4470-49e9-bea3-e44335d3b2f7 service placement] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:40.844805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:40.849670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:40.851206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:40.852659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] PATH_INFO: `/v3/users/aabdbb0cb065463bb778db01ce2f004a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:40.855329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabdbb0cb065463bb778db01ce2f004a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:40.859543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:40.995549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabdbb0cb065463bb778db01ce2f004a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.001340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36001551-09a4-4573-8277-7f62852c92df service nova] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:41.015502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:41.016737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:41.018502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:41.019361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] PATH_INFO: `/v3/users/dd95af9b9bc246d7aa6445e25e39a7db` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:41.023986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd95af9b9bc246d7aa6445e25e39a7db)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:41.029852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:41.033727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabdbb0cb065463bb778db01ce2f004a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.051314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabdbb0cb065463bb778db01ce2f004a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.064006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabdbb0cb065463bb778db01ce2f004a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.068795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-167dbbba-84c1-4be9-b61f-1373c5503e8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabdbb0cb065463bb778db01ce2f004a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.104860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 844/1710] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:40 2022] DELETE /identity/v3/users/aabdbb0cb065463bb778db01ce2f004a => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:41.106977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4efd5-8688-4d1e-ab8c-682a41457a56 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:41.106977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4efd5-8688-4d1e-ab8c-682a41457a56 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:41.107186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4efd5-8688-4d1e-ab8c-682a41457a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:41.160082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f5d4efd5-8688-4d1e-ab8c-682a41457a56 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:41.161579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f5d4efd5-8688-4d1e-ab8c-682a41457a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:41.182571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd95af9b9bc246d7aa6445e25e39a7db'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.220907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:41.226845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd95af9b9bc246d7aa6445e25e39a7db'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.247544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 845/1711] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:41.252058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd95af9b9bc246d7aa6445e25e39a7db'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.252716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd95af9b9bc246d7aa6445e25e39a7db'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.266210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd95af9b9bc246d7aa6445e25e39a7db'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:41.284705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 867/1712] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:40 2022] DELETE /identity/v3/users/dd95af9b9bc246d7aa6445e25e39a7db => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:41.290330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17e463-b63f-4746-b708-5a02e92ef5fb None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:41.290637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17e463-b63f-4746-b708-5a02e92ef5fb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:41.290929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17e463-b63f-4746-b708-5a02e92ef5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:41.303262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:41.305481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:41.343958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-9a17e463-b63f-4746-b708-5a02e92ef5fb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:41.352485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9a17e463-b63f-4746-b708-5a02e92ef5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:41.509130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81d461-8088-492c-8d4e-92b426ee156e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:41.531070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:41.532025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:41.533956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:41.534340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:41.537063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:41.540693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 846/1713] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:53:41.587696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:41.592576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:41.640552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17e463-b63f-4746-b708-5a02e92ef5fb None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:41.656698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 868/1714] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:41.663969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17e463-b63f-4746-b708-5a02e92ef5fb None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:41.678781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:41.681916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:41.683432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:41.683719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:41.719729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:41.727664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:41.762967 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:41.795197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 847/1715] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:53:42.819134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:42.821259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:42.844570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df07c-f0ab-4bc6-9b46-f6d1f7514d98 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:42.861899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:42.863035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:42.863456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:42.863915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] PATH_INFO: `/v3/users/76f88f8b791743e3a6b7cb0d29f3b4f6` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:42.869081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f88f8b791743e3a6b7cb0d29f3b4f6)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:42.871759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:42.936076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:42.941945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:42.979890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f88f8b791743e3a6b7cb0d29f3b4f6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:42.990026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667c853-6e67-4734-ac46-776ab43f79e6 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:43.015745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:43.020412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:43.021249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:43.023953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] PATH_INFO: `/v3/projects/8a626149a0ee4b3db0113c7084b37cfa` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:43.026440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f88f8b791743e3a6b7cb0d29f3b4f6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.028896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a626149a0ee4b3db0113c7084b37cfa)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:43.036990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f88f8b791743e3a6b7cb0d29f3b4f6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.037949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f88f8b791743e3a6b7cb0d29f3b4f6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.041985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:43.055978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f88f8b791743e3a6b7cb0d29f3b4f6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.067216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 869/1716] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:42 2022] DELETE /identity/v3/users/76f88f8b791743e3a6b7cb0d29f3b4f6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:43.274093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:43.275571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:43.288903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a626149a0ee4b3db0113c7084b37cfa', 'request_id': 'req-6ef70368-c9de-4d48-bd47-d640b3f316de'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.320287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a626149a0ee4b3db0113c7084b37cfa', 'request_id': 'req-6ef70368-c9de-4d48-bd47-d640b3f316de'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.341696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a626149a0ee4b3db0113c7084b37cfa', 'request_id': 'req-6ef70368-c9de-4d48-bd47-d640b3f316de'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.370922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] The token cache is being invalidate because project 8a626149a0ee4b3db0113c7084b37cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:43.371238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.373593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6ef70368-c9de-4d48-bd47-d640b3f316de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.377894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 848/1717] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:42 2022] DELETE /identity/v3/projects/8a626149a0ee4b3db0113c7084b37cfa => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:53:43.383764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d94438-8b0e-402d-8d0c-f1514e065780 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:43.384051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d94438-8b0e-402d-8d0c-f1514e065780 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:43.384330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d94438-8b0e-402d-8d0c-f1514e065780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:43.492654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:43.516481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e76417-6287-4cc9-bd50-9d71790931ae admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:43.521399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-74d94438-8b0e-402d-8d0c-f1514e065780 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:43.538703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-74d94438-8b0e-402d-8d0c-f1514e065780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:43.699126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 849/1718] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:43.710664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ef52-d80e-4b7e-a31b-29ddcabe0a75 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:43.711081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ef52-d80e-4b7e-a31b-29ddcabe0a75 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:43.711428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ef52-d80e-4b7e-a31b-29ddcabe0a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:43.749917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:43.751524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:43.751812 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:43.752107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] PATH_INFO: `/v3/projects/b9bf6d48896d42a3b45ff20a9790f79b` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:43.757124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9bf6d48896d42a3b45ff20a9790f79b)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:43.759821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:43.761234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2376ef52-d80e-4b7e-a31b-29ddcabe0a75 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:43.762739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2376ef52-d80e-4b7e-a31b-29ddcabe0a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:43.781034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 850/1719] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:43.787044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376ef52-d80e-4b7e-a31b-29ddcabe0a75 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:43.808758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376ef52-d80e-4b7e-a31b-29ddcabe0a75 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:43.819376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:43.821090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:43.821382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:43.821662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:43.824672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:43.824867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:43.831053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:43.861269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 851/1720] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:43.867507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:43.872335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9bf6d48896d42a3b45ff20a9790f79b', 'request_id': 'req-1689909c-035a-47df-9a4d-09eac2de7978'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.891171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9bf6d48896d42a3b45ff20a9790f79b', 'request_id': 'req-1689909c-035a-47df-9a4d-09eac2de7978'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.901553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ee4f8-27e6-49f2-b6fb-fa9d2d20048b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:43.908673 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9bf6d48896d42a3b45ff20a9790f79b', 'request_id': 'req-1689909c-035a-47df-9a4d-09eac2de7978'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.937426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] The token cache is being invalidate because project b9bf6d48896d42a3b45ff20a9790f79b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:43.937729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.940652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:43.950604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 870/1721] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:43 2022] DELETE /identity/v3/projects/b9bf6d48896d42a3b45ff20a9790f79b => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:43.950604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:43.950604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:44.071960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:44.072808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:44.073320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:44.073707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:44.075337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:44.084383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:44.101281 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:44.148824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689909c-035a-47df-9a4d-09eac2de7978 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:44.151068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 852/1722] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:43 2022] POST /identity/v3/projects => generated 373 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:44.162386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:44.165824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:44.293450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:44.383836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:44.383836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:44.383836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:44.383836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] PATH_INFO: `/v3/projects/0a8326517d0f4bc191b63059f6b5721b` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:44.385179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8326517d0f4bc191b63059f6b5721b)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:44.387726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:44.399947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dee914-72d1-4403-9677-adabfbc6539d admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:44.419230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:44.420146 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:44.420560 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:44.420867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] PATH_INFO: `/v3/users/68fe9655abf34c27945519cdffc89a6a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:44.422118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] RBAC: Authorizing `identity:delete_user(user_id=68fe9655abf34c27945519cdffc89a6a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:44.424092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:44.589540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68fe9655abf34c27945519cdffc89a6a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.629156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8326517d0f4bc191b63059f6b5721b', 'request_id': 'req-03a76f32-ca31-4ca8-8a32-3df1b6891b62'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.629882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68fe9655abf34c27945519cdffc89a6a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.651664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68fe9655abf34c27945519cdffc89a6a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.661028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8326517d0f4bc191b63059f6b5721b', 'request_id': 'req-03a76f32-ca31-4ca8-8a32-3df1b6891b62'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.675446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68fe9655abf34c27945519cdffc89a6a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.676585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68fe9655abf34c27945519cdffc89a6a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.680895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8326517d0f4bc191b63059f6b5721b', 'request_id': 'req-03a76f32-ca31-4ca8-8a32-3df1b6891b62'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.686288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 853/1723] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:44 2022] DELETE /identity/v3/users/68fe9655abf34c27945519cdffc89a6a => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:44.688103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:44.693896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:44.717668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] The token cache is being invalidate because project 0a8326517d0f4bc191b63059f6b5721b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:44.718043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.718732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-03a76f32-ca31-4ca8-8a32-3df1b6891b62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:44.723359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 871/1724] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:43 2022] DELETE /identity/v3/projects/0a8326517d0f4bc191b63059f6b5721b => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:44.723359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d156d0a-0d0f-4532-871b-695c85da9f40 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:44.723359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d156d0a-0d0f-4532-871b-695c85da9f40 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:44.723359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d156d0a-0d0f-4532-871b-695c85da9f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:44.796359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6d156d0a-0d0f-4532-871b-695c85da9f40 None None] MFA Rules not processed for user `9a67a5c4b7034d22b78331b907cc20bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:44.797824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6d156d0a-0d0f-4532-871b-695c85da9f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:44.899302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8306de-d72d-40d8-adfc-a347f92d815a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:44.911577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:44.914487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:44.915152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:44.915684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:44.916797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:44.950363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:44.993412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:45.008253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 854/1725] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:44 2022] POST /identity/v3/users => generated 287 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:45.010012 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:45.012380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:45.034041 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a809ef-50fb-4dbf-984a-e4e755f36df9 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:45.038248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 872/1726] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:45.039357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d156d0a-0d0f-4532-871b-695c85da9f40 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:45.041332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6d156d0a-0d0f-4532-871b-695c85da9f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:45.055969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:45.057018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:45.057207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:45.057460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] PATH_INFO: `/v3/users/31219f8bb890473588d968c33b170e6f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:45.063608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31219f8bb890473588d968c33b170e6f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:45.067304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:45.077473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d156d0a-0d0f-4532-871b-695c85da9f40 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:45.092241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:45.092992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:45.094420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:45.094829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:45.096193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:45.100546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:45.134294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 873/1727] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:45.134294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:45.138034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:45.187513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31219f8bb890473588d968c33b170e6f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.218055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31219f8bb890473588d968c33b170e6f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.224145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:45.233501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31219f8bb890473588d968c33b170e6f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.250367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31219f8bb890473588d968c33b170e6f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.252386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31219f8bb890473588d968c33b170e6f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.264499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 855/1728] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:45 2022] DELETE /identity/v3/users/31219f8bb890473588d968c33b170e6f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:45.265810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:45.267236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:45.272193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad49b5-a029-4da8-a9fa-f9691ec4e023 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:45.375037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:45.376191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:45.376811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:45.377305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:45.380724 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:45.443852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1817095-1408-48b6-aa66-9e6b6a426a42 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:45.456520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:45.458377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:45.459131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:45.460393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:45.461713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:45.467262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:45.486066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:45.488201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:45.492433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 856/1729] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:45.497729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:45.502542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:45.518807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 874/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:45.525066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:45.528104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8d81a-34bb-4145-998b-5cb81f6f8d4d admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:45.533436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:45.534519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:45.534869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:45.535500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] PATH_INFO: `/v3/projects/741114fb60f5466f85ee8f4beefc7ab2` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:45.538053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=741114fb60f5466f85ee8f4beefc7ab2)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:45.540360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:45.560324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ee48c-053e-4bb6-803f-243a8c86090d service nova] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:45.578316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5b149-83fb-48cd-9574-eb004da13290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:45.579372 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5b149-83fb-48cd-9574-eb004da13290 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:45.579658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5b149-83fb-48cd-9574-eb004da13290 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:45.579962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5b149-83fb-48cd-9574-eb004da13290 admin admin] PATH_INFO: `/v3/projects/bc9f7f1f0a754ab9b2f6e46c842382e3/users/9c1a6ed1058e48dc95d675110f6ae98e/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:45.617837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5b149-83fb-48cd-9574-eb004da13290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9f7f1f0a754ab9b2f6e46c842382e3, user_id=9c1a6ed1058e48dc95d675110f6ae98e, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:45.625895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5b149-83fb-48cd-9574-eb004da13290 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:45.665577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 875/1731] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:45 2022] PUT /identity/v3/projects/bc9f7f1f0a754ab9b2f6e46c842382e3/users/9c1a6ed1058e48dc95d675110f6ae98e/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:45.669559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9b435-6c39-48b3-ba39-879b6816320e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:45.669905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9b435-6c39-48b3-ba39-879b6816320e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:45.670190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9b435-6c39-48b3-ba39-879b6816320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:45.697402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741114fb60f5466f85ee8f4beefc7ab2', 'request_id': 'req-3f3f8274-0048-4dec-bcf3-5b1233748c96'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.722334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741114fb60f5466f85ee8f4beefc7ab2', 'request_id': 'req-3f3f8274-0048-4dec-bcf3-5b1233748c96'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.735667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2fb9b435-6c39-48b3-ba39-879b6816320e None None] MFA Rules not processed for user `9c1a6ed1058e48dc95d675110f6ae98e`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:45.737658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2fb9b435-6c39-48b3-ba39-879b6816320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:45.748721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741114fb60f5466f85ee8f4beefc7ab2', 'request_id': 'req-3f3f8274-0048-4dec-bcf3-5b1233748c96'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.752630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:45.791683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] The token cache is being invalidate because project 741114fb60f5466f85ee8f4beefc7ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:45.793463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.798312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:45.802145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 857/1732] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:45 2022] DELETE /identity/v3/projects/741114fb60f5466f85ee8f4beefc7ab2 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:45.807481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:45.809856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:45.834210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:45.913039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 876/1733] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:45.917440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705f54-72a9-4af8-b403-ae21dab59310 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:45.918313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705f54-72a9-4af8-b403-ae21dab59310 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:45.919663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705f54-72a9-4af8-b403-ae21dab59310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:45.960427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-78705f54-72a9-4af8-b403-ae21dab59310 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:45.961659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-78705f54-72a9-4af8-b403-ae21dab59310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:46.010848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f8274-0048-4dec-bcf3-5b1233748c96 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:46.054041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 877/1734] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:46.055332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78705f54-72a9-4af8-b403-ae21dab59310 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:46.057269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-78705f54-72a9-4af8-b403-ae21dab59310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:46.148368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:46.149297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.149552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.149848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] PATH_INFO: `/v3/projects/e7ec32cfb4b843309d5983886dd6fb8d` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.151328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ec32cfb4b843309d5983886dd6fb8d)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:46.154723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:46.216414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78705f54-72a9-4af8-b403-ae21dab59310 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:46.238139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f16b4-11a2-4fb1-a793-3e4e590d638d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:46.240065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f16b4-11a2-4fb1-a793-3e4e590d638d service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.241577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f16b4-11a2-4fb1-a793-3e4e590d638d service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.241892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f16b4-11a2-4fb1-a793-3e4e590d638d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.263211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f16b4-11a2-4fb1-a793-3e4e590d638d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:46.274551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ec32cfb4b843309d5983886dd6fb8d', 'request_id': 'req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:46.276306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f16b4-11a2-4fb1-a793-3e4e590d638d service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:46.286607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ec32cfb4b843309d5983886dd6fb8d', 'request_id': 'req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:46.302104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ec32cfb4b843309d5983886dd6fb8d', 'request_id': 'req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:46.304709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 878/1735] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:46.307444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1b34-91d2-4de8-aec7-34e178fe88be None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.307816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1b34-91d2-4de8-aec7-34e178fe88be None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.308091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e1b34-91d2-4de8-aec7-34e178fe88be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.327872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] The token cache is being invalidate because project e7ec32cfb4b843309d5983886dd6fb8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:46.328164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:46.328838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3d4a76ef-1921-43c7-b01f-7185ed6cacc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:46.333021 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 858/1736] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:45 2022] DELETE /identity/v3/projects/e7ec32cfb4b843309d5983886dd6fb8d => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:46.338062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853707c-317e-4569-a47d-da310d13c726 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.338350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853707c-317e-4569-a47d-da310d13c726 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.338631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853707c-317e-4569-a47d-da310d13c726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.353007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1f5e1b34-91d2-4de8-aec7-34e178fe88be None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:46.354395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1f5e1b34-91d2-4de8-aec7-34e178fe88be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:46.370781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8853707c-317e-4569-a47d-da310d13c726 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:46.373387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8853707c-317e-4569-a47d-da310d13c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:46.415120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:46.465691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 879/1737] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:46.471942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e1b34-91d2-4de8-aec7-34e178fe88be None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:46.483726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e1b34-91d2-4de8-aec7-34e178fe88be None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:46.490891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 859/1738] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:46.498172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b89fa-2bc2-4e5e-b33d-c83b138d95e4 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.498397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:46.499093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.499379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.499654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.500681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:46.500994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:46.502262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b89fa-2bc2-4e5e-b33d-c83b138d95e4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.502644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:46.502891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b89fa-2bc2-4e5e-b33d-c83b138d95e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.523296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 880/1739] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:53:46.534179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:46.538364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-261b89fa-2bc2-4e5e-b33d-c83b138d95e4 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:46.539771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-261b89fa-2bc2-4e5e-b33d-c83b138d95e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:46.550088 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:46.559651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 860/1740] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:46.562957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b89fa-2bc2-4e5e-b33d-c83b138d95e4 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:46.566170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79394799-514a-4658-8624-ca45dfe7645c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:46.580644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53a2b-faaa-4a3e-9bce-65bec3e6616a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:46.580947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b89fa-2bc2-4e5e-b33d-c83b138d95e4 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:46.582646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53a2b-faaa-4a3e-9bce-65bec3e6616a admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.582938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53a2b-faaa-4a3e-9bce-65bec3e6616a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.583220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e53a2b-faaa-4a3e-9bce-65bec3e6616a admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.586165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53a2b-faaa-4a3e-9bce-65bec3e6616a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:46.590089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e53a2b-faaa-4a3e-9bce-65bec3e6616a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:46.592141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:46.593106 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.593569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.594067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.595121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:46.595507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:46.598244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:46.634889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 861/1741] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:53:46.639490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:46.663981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 881/1742] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:46 2022] POST /identity/v3/projects => generated 393 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:46.672639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53a2b-faaa-4a3e-9bce-65bec3e6616a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:46.685192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1d985-7034-4373-b1cc-5ce444d18380 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:46.909195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e53a2b-faaa-4a3e-9bce-65bec3e6616a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:46.927864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287925d2-c220-4b63-bb06-b8566fcbd234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:46.928862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287925d2-c220-4b63-bb06-b8566fcbd234 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:46.929630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287925d2-c220-4b63-bb06-b8566fcbd234 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:46.930098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287925d2-c220-4b63-bb06-b8566fcbd234 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:46.932070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287925d2-c220-4b63-bb06-b8566fcbd234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:46.934745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287925d2-c220-4b63-bb06-b8566fcbd234 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.056496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 882/1743] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:46 2022] POST /identity/v3/users => generated 297 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:47.056496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287925d2-c220-4b63-bb06-b8566fcbd234 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:47.089577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287925d2-c220-4b63-bb06-b8566fcbd234 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:47.101338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750d54-84b1-47cb-9ebc-e15bd1bdc6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:47.102185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750d54-84b1-47cb-9ebc-e15bd1bdc6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.102533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750d54-84b1-47cb-9ebc-e15bd1bdc6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.102892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed750d54-84b1-47cb-9ebc-e15bd1bdc6c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.103268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e15a-64cf-4e57-9218-6fc51346c866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:47.103752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed750d54-84b1-47cb-9ebc-e15bd1bdc6c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:47.105491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e15a-64cf-4e57-9218-6fc51346c866 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.106534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e15a-64cf-4e57-9218-6fc51346c866 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.107532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e15a-64cf-4e57-9218-6fc51346c866 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.108725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed750d54-84b1-47cb-9ebc-e15bd1bdc6c2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.112480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7e15a-64cf-4e57-9218-6fc51346c866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:47.118201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7e15a-64cf-4e57-9218-6fc51346c866 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.174680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 862/1744] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:46 2022] POST /identity/v3/projects => generated 365 bytes in 537 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:47.177949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 883/1745] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:47.181488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750d54-84b1-47cb-9ebc-e15bd1bdc6c2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:47.181850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e15a-64cf-4e57-9218-6fc51346c866 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:47.203531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e15a-64cf-4e57-9218-6fc51346c866 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:47.349834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed750d54-84b1-47cb-9ebc-e15bd1bdc6c2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:47.357216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43592a90-7d30-420e-98b9-3219b6c81e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:47.358519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43592a90-7d30-420e-98b9-3219b6c81e22 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.359133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43592a90-7d30-420e-98b9-3219b6c81e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.360100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43592a90-7d30-420e-98b9-3219b6c81e22 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.361307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43592a90-7d30-420e-98b9-3219b6c81e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:47.364028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43592a90-7d30-420e-98b9-3219b6c81e22 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.391328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2495f0-45db-4a88-8dad-6f84ab295c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:47.391921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2495f0-45db-4a88-8dad-6f84ab295c61 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.392209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2495f0-45db-4a88-8dad-6f84ab295c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.392490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2495f0-45db-4a88-8dad-6f84ab295c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.393227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2495f0-45db-4a88-8dad-6f84ab295c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:47.407898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 863/1746] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:47 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:47.414392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2495f0-45db-4a88-8dad-6f84ab295c61 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.422326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43592a90-7d30-420e-98b9-3219b6c81e22 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:47.443516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 884/1747] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:47.451951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2495f0-45db-4a88-8dad-6f84ab295c61 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:47.456852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43592a90-7d30-420e-98b9-3219b6c81e22 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:47.472025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2495f0-45db-4a88-8dad-6f84ab295c61 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:47.482235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3befd86-fc18-40f4-b460-48d77bf9eae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:47.483017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3befd86-fc18-40f4-b460-48d77bf9eae0 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.483301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3befd86-fc18-40f4-b460-48d77bf9eae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.483578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3befd86-fc18-40f4-b460-48d77bf9eae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.484344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3befd86-fc18-40f4-b460-48d77bf9eae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:47.486288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3befd86-fc18-40f4-b460-48d77bf9eae0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.509456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b56237-70a9-447b-b31e-9596504d6969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:47.510329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 864/1748] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:47.511099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b56237-70a9-447b-b31e-9596504d6969 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.511639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3befd86-fc18-40f4-b460-48d77bf9eae0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:47.512252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b56237-70a9-447b-b31e-9596504d6969 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.512808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b56237-70a9-447b-b31e-9596504d6969 admin admin] PATH_INFO: `/v3/projects/b8ded3e9f6d24612a29985564f1ea7eb/users/b0f67d2032284d0f93984ab05f2233d7/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.536457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3befd86-fc18-40f4-b460-48d77bf9eae0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:47.547462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eedbe-88d7-4252-adbe-c436e05b55ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:47.548168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eedbe-88d7-4252-adbe-c436e05b55ed admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.548456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eedbe-88d7-4252-adbe-c436e05b55ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.548731 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0eedbe-88d7-4252-adbe-c436e05b55ed admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.549465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0eedbe-88d7-4252-adbe-c436e05b55ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:47.551127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0eedbe-88d7-4252-adbe-c436e05b55ed admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.563917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:47.565874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b56237-70a9-447b-b31e-9596504d6969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ded3e9f6d24612a29985564f1ea7eb, user_id=b0f67d2032284d0f93984ab05f2233d7, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:47.576212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b56237-70a9-447b-b31e-9596504d6969 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.583326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 865/1749] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:47.584967 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eedbe-88d7-4252-adbe-c436e05b55ed admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:47.601391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 885/1750] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:47 2022] PUT /identity/v3/projects/b8ded3e9f6d24612a29985564f1ea7eb/users/b0f67d2032284d0f93984ab05f2233d7/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:47.620746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a580-2e92-41ee-951f-0ef0a5c0258d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.621322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a580-2e92-41ee-951f-0ef0a5c0258d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.621822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a580-2e92-41ee-951f-0ef0a5c0258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.627995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0eedbe-88d7-4252-adbe-c436e05b55ed admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:47.702094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-0c38a580-2e92-41ee-951f-0ef0a5c0258d None None] MFA Rules not processed for user `b0f67d2032284d0f93984ab05f2233d7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:47.703341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0c38a580-2e92-41ee-951f-0ef0a5c0258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:47.758913 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:47.795133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0e4d3-7f53-4799-8849-cfbe490a99e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:47.796200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e4d3-7f53-4799-8849-cfbe490a99e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.796517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e4d3-7f53-4799-8849-cfbe490a99e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.796850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0e4d3-7f53-4799-8849-cfbe490a99e4 admin admin] PATH_INFO: `/v3/projects/20fac0197749438b8665b82cd70b6890/users/2e36d610bb1447b0a06e6764780357b3/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.838574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0e4d3-7f53-4799-8849-cfbe490a99e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fac0197749438b8665b82cd70b6890, user_id=2e36d610bb1447b0a06e6764780357b3, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:47.841317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0e4d3-7f53-4799-8849-cfbe490a99e4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:47.883973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 866/1751] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:47 2022] PUT /identity/v3/projects/20fac0197749438b8665b82cd70b6890/users/2e36d610bb1447b0a06e6764780357b3/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:47.886512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc98ea-3ae2-495b-aca9-8bed96cc9199 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:47.887573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc98ea-3ae2-495b-aca9-8bed96cc9199 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:47.889224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc98ea-3ae2-495b-aca9-8bed96cc9199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:47.969878 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f7fc98ea-3ae2-495b-aca9-8bed96cc9199 None None] MFA Rules not processed for user `2e36d610bb1447b0a06e6764780357b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:47.971449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f7fc98ea-3ae2-495b-aca9-8bed96cc9199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:48.100790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 867/1752] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:48.102378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc98ea-3ae2-495b-aca9-8bed96cc9199 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.111278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 886/1753] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:48.112540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a580-2e92-41ee-951f-0ef0a5c0258d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.127091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a580-2e92-41ee-951f-0ef0a5c0258d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:48.201112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5231a4-639f-4b89-9c63-4a5c80fcfafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:48.201820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5231a4-639f-4b89-9c63-4a5c80fcfafe admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:48.202106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5231a4-639f-4b89-9c63-4a5c80fcfafe admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:48.202381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5231a4-639f-4b89-9c63-4a5c80fcfafe admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:48.203082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5231a4-639f-4b89-9c63-4a5c80fcfafe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:48.205030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5231a4-639f-4b89-9c63-4a5c80fcfafe admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:48.237434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 887/1754] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:48 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:48.246713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5231a4-639f-4b89-9c63-4a5c80fcfafe admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.273249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc98ea-3ae2-495b-aca9-8bed96cc9199 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:48.296643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:48.392948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281e4ca-e675-44d4-9c50-397d55e1a8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:48.393692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281e4ca-e675-44d4-9c50-397d55e1a8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:48.395270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281e4ca-e675-44d4-9c50-397d55e1a8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:48.395596 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281e4ca-e675-44d4-9c50-397d55e1a8c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:48.398423 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c281e4ca-e675-44d4-9c50-397d55e1a8c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:48.402786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c281e4ca-e675-44d4-9c50-397d55e1a8c6 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:48.436843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5231a4-639f-4b89-9c63-4a5c80fcfafe admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:48.457239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8900d5-c978-487c-a8b5-669baa919bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:48.459589 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8900d5-c978-487c-a8b5-669baa919bcf admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:48.461822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8900d5-c978-487c-a8b5-669baa919bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:48.462146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8900d5-c978-487c-a8b5-669baa919bcf admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:48.466261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8900d5-c978-487c-a8b5-669baa919bcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:48.473471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8900d5-c978-487c-a8b5-669baa919bcf admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:48.474364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 868/1755] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:48 2022] POST /identity/v3/projects => generated 363 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:48.480232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281e4ca-e675-44d4-9c50-397d55e1a8c6 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.519767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 888/1756] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:48 2022] POST /identity/v3/users => generated 303 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:48.523735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8900d5-c978-487c-a8b5-669baa919bcf admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.547281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8900d5-c978-487c-a8b5-669baa919bcf admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:48.622094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281e4ca-e675-44d4-9c50-397d55e1a8c6 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:48.634775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786739f-927e-4e61-b93e-eb2e92fd11e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:48.635786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786739f-927e-4e61-b93e-eb2e92fd11e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:48.636969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786739f-927e-4e61-b93e-eb2e92fd11e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:48.637414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786739f-927e-4e61-b93e-eb2e92fd11e0 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:48.638485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786739f-927e-4e61-b93e-eb2e92fd11e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:48.640841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786739f-927e-4e61-b93e-eb2e92fd11e0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:48.643715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:48.696913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 869/1757] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:48 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:48.700875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786739f-927e-4e61-b93e-eb2e92fd11e0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.718749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3e6a6-8eb1-465d-a3f1-53658e2637d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:48.719485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e6a6-8eb1-465d-a3f1-53658e2637d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:48.719775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e6a6-8eb1-465d-a3f1-53658e2637d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:48.720054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3e6a6-8eb1-465d-a3f1-53658e2637d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:48.720870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3e6a6-8eb1-465d-a3f1-53658e2637d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:48.722897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786739f-927e-4e61-b93e-eb2e92fd11e0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:48.723057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3e6a6-8eb1-465d-a3f1-53658e2637d3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:48.736901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62dc4-3c5c-4bd0-8e3f-a185a9fcb2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:48.737753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62dc4-3c5c-4bd0-8e3f-a185a9fcb2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:48.738148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62dc4-3c5c-4bd0-8e3f-a185a9fcb2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:48.738517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62dc4-3c5c-4bd0-8e3f-a185a9fcb2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:48.739419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb62dc4-3c5c-4bd0-8e3f-a185a9fcb2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:48.741978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb62dc4-3c5c-4bd0-8e3f-a185a9fcb2b8 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:48.751195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 889/1758] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:48.752192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3e6a6-8eb1-465d-a3f1-53658e2637d3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.781701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3e6a6-8eb1-465d-a3f1-53658e2637d3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:48.798892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20217f61-2883-4b14-a9a9-1f9005ccade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:48.799067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 870/1759] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:48.800089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217f61-2883-4b14-a9a9-1f9005ccade9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:48.800389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217f61-2883-4b14-a9a9-1f9005ccade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:48.800667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20217f61-2883-4b14-a9a9-1f9005ccade9 admin admin] PATH_INFO: `/v3/projects/fe3c33a627154a87b3317cdd1d4c11f7/users/7f28af369cac4351b9350bd8317fa2a3/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:48.801807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62dc4-3c5c-4bd0-8e3f-a185a9fcb2b8 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.838668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20217f61-2883-4b14-a9a9-1f9005ccade9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3c33a627154a87b3317cdd1d4c11f7, user_id=7f28af369cac4351b9350bd8317fa2a3, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:48.841390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20217f61-2883-4b14-a9a9-1f9005ccade9 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:48.855242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62dc4-3c5c-4bd0-8e3f-a185a9fcb2b8 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:48.869101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7062c56-7e9e-4cb8-9340-a19740c004c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:48.869330 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 890/1760] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:48 2022] PUT /identity/v3/projects/fe3c33a627154a87b3317cdd1d4c11f7/users/7f28af369cac4351b9350bd8317fa2a3/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:48.873585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7062c56-7e9e-4cb8-9340-a19740c004c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:48.874259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7062c56-7e9e-4cb8-9340-a19740c004c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:48.875718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7062c56-7e9e-4cb8-9340-a19740c004c5 admin admin] PATH_INFO: `/v3/projects/dcf51e917d4f448fa7ed5e1b77cb4f4e/users/8be157c0ba61493eade91da1d0ab2688/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:48.876688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20217f61-2883-4b14-a9a9-1f9005ccade9 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:48.931927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7062c56-7e9e-4cb8-9340-a19740c004c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf51e917d4f448fa7ed5e1b77cb4f4e, user_id=8be157c0ba61493eade91da1d0ab2688, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:48.935001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7062c56-7e9e-4cb8-9340-a19740c004c5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:48.970710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 871/1761] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:48 2022] PUT /identity/v3/projects/dcf51e917d4f448fa7ed5e1b77cb4f4e/users/8be157c0ba61493eade91da1d0ab2688/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:48.975009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7062c56-7e9e-4cb8-9340-a19740c004c5 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:49.104141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20217f61-2883-4b14-a9a9-1f9005ccade9 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:49.141288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7062c56-7e9e-4cb8-9340-a19740c004c5 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:49.158725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf7111-d7c5-4a74-b21f-3d9833ea25e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:49.159425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7111-d7c5-4a74-b21f-3d9833ea25e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:49.159707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7111-d7c5-4a74-b21f-3d9833ea25e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:49.159999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cf7111-d7c5-4a74-b21f-3d9833ea25e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:49.160719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf7111-d7c5-4a74-b21f-3d9833ea25e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:49.163120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cf7111-d7c5-4a74-b21f-3d9833ea25e2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:49.174043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:49.203978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 872/1762] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:49.205048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf7111-d7c5-4a74-b21f-3d9833ea25e2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:49.231791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cf7111-d7c5-4a74-b21f-3d9833ea25e2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:49.240606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474f4b5-89d9-4195-8461-fe0c8ce63347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:49.241807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474f4b5-89d9-4195-8461-fe0c8ce63347 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:49.246253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474f4b5-89d9-4195-8461-fe0c8ce63347 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:49.246719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474f4b5-89d9-4195-8461-fe0c8ce63347 admin admin] PATH_INFO: `/v3/projects/dcf51e917d4f448fa7ed5e1b77cb4f4e/users/8be157c0ba61493eade91da1d0ab2688/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:49.249447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474f4b5-89d9-4195-8461-fe0c8ce63347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf51e917d4f448fa7ed5e1b77cb4f4e, user_id=8be157c0ba61493eade91da1d0ab2688, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:49.255074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474f4b5-89d9-4195-8461-fe0c8ce63347 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:49.283186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 873/1763] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:49 2022] PUT /identity/v3/projects/dcf51e917d4f448fa7ed5e1b77cb4f4e/users/8be157c0ba61493eade91da1d0ab2688/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:49.297657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e1b3-c4c7-4046-94f3-b18f3b629cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:49.298267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3733920-cfe8-4243-a8a7-686f32ff715c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:49.299179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e1b3-c4c7-4046-94f3-b18f3b629cdf admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:49.299566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3733920-cfe8-4243-a8a7-686f32ff715c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:49.299883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e1b3-c4c7-4046-94f3-b18f3b629cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:49.300197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3733920-cfe8-4243-a8a7-686f32ff715c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:49.300466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935e1b3-c4c7-4046-94f3-b18f3b629cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:49.301410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e1b3-c4c7-4046-94f3-b18f3b629cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:49.306227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935e1b3-c4c7-4046-94f3-b18f3b629cdf admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:49.356998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 891/1764] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:49.358243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e1b3-c4c7-4046-94f3-b18f3b629cdf admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:49.403741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935e1b3-c4c7-4046-94f3-b18f3b629cdf admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:49.406051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b3733920-cfe8-4243-a8a7-686f32ff715c None None] MFA Rules not processed for user `8be157c0ba61493eade91da1d0ab2688`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:49.408490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b3733920-cfe8-4243-a8a7-686f32ff715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:49.549041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:49.582500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bdeec-f126-49e2-9cac-6b1079ccde65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:49.584050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bdeec-f126-49e2-9cac-6b1079ccde65 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:49.584676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bdeec-f126-49e2-9cac-6b1079ccde65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:49.585294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bdeec-f126-49e2-9cac-6b1079ccde65 admin admin] PATH_INFO: `/v3/projects/fe3c33a627154a87b3317cdd1d4c11f7/users/7f28af369cac4351b9350bd8317fa2a3/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:49.591692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bdeec-f126-49e2-9cac-6b1079ccde65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3c33a627154a87b3317cdd1d4c11f7, user_id=7f28af369cac4351b9350bd8317fa2a3, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:49.596931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bdeec-f126-49e2-9cac-6b1079ccde65 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:49.621594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 874/1765] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:49 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:49.632972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3733920-cfe8-4243-a8a7-686f32ff715c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:49.641609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 892/1766] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:49 2022] PUT /identity/v3/projects/fe3c33a627154a87b3317cdd1d4c11f7/users/7f28af369cac4351b9350bd8317fa2a3/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:49.642065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b3733920-cfe8-4243-a8a7-686f32ff715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:49.644804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bd032-a4a6-4a31-a939-7364e663d7b9 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:49.645096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bd032-a4a6-4a31-a939-7364e663d7b9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:49.645377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6bd032-a4a6-4a31-a939-7364e663d7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:49.699676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ed6bd032-a4a6-4a31-a939-7364e663d7b9 None None] MFA Rules not processed for user `7f28af369cac4351b9350bd8317fa2a3`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:49.703394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ed6bd032-a4a6-4a31-a939-7364e663d7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:49.846403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3733920-cfe8-4243-a8a7-686f32ff715c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:49.938848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 893/1767] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:49.962745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef5f1e-aa63-4058-94c8-415c7b8f4f62 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:49.963050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef5f1e-aa63-4058-94c8-415c7b8f4f62 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:49.963874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef5f1e-aa63-4058-94c8-415c7b8f4f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.002589 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-77ef5f1e-aa63-4058-94c8-415c7b8f4f62 None None] MFA Rules not processed for user `fa5a48626c934dc2aa4f24002970b6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:50.004986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-77ef5f1e-aa63-4058-94c8-415c7b8f4f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.021726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:50.022479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:50.022788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:50.023068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.153018 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:50.174333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 894/1768] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:50.175819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc61265-d314-41a0-b2d7-e5e65b200d13 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:50.176800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc61265-d314-41a0-b2d7-e5e65b200d13 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:50.177089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc61265-d314-41a0-b2d7-e5e65b200d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.210825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:50.212489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:50.232194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1fc61265-d314-41a0-b2d7-e5e65b200d13 None None] MFA Rules not processed for user `75407a21b1fd46e8ba10540fc5a4c30a`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:50.241045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1fc61265-d314-41a0-b2d7-e5e65b200d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.247811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 875/1769] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:49 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:50.248845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:50.253232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.295234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672ecc3-3f81-45d8-8eb5-bc9ec508db95 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:50.308824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10042058-83a4-43d0-a553-24c096536781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:50.309869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10042058-83a4-43d0-a553-24c096536781 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:50.310250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10042058-83a4-43d0-a553-24c096536781 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:50.311143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10042058-83a4-43d0-a553-24c096536781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.326036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10042058-83a4-43d0-a553-24c096536781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:50.328599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10042058-83a4-43d0-a553-24c096536781 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:50.342118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 876/1770] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:50.343768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2be3e-e2b2-4618-9b1f-1b591e05ede5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:50.344316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2be3e-e2b2-4618-9b1f-1b591e05ede5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:50.344809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2be3e-e2b2-4618-9b1f-1b591e05ede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.424125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 895/1771] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:50.425283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc61265-d314-41a0-b2d7-e5e65b200d13 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:50.430020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1fc61265-d314-41a0-b2d7-e5e65b200d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.444035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:50.511085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1db2be3e-e2b2-4618-9b1f-1b591e05ede5 None None] MFA Rules not processed for user `23031b94dc7f47bf8272b266cbe18a8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:50.517279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1db2be3e-e2b2-4618-9b1f-1b591e05ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.527560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc61265-d314-41a0-b2d7-e5e65b200d13 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:50.532521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:50.533636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:50.534042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:50.534418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] PATH_INFO: `/v3/users/9c1a6ed1058e48dc95d675110f6ae98e` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.536804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1a6ed1058e48dc95d675110f6ae98e)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:50.539315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:50.622087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1a6ed1058e48dc95d675110f6ae98e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:50.639691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1a6ed1058e48dc95d675110f6ae98e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:50.651755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1a6ed1058e48dc95d675110f6ae98e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:50.652741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1a6ed1058e48dc95d675110f6ae98e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:50.677559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1a6ed1058e48dc95d675110f6ae98e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:50.698775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 896/1772] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:50 2022] DELETE /identity/v3/users/9c1a6ed1058e48dc95d675110f6ae98e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:50.700993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:50.705825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.756561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 877/1773] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:50.757625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2be3e-e2b2-4618-9b1f-1b591e05ede5 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:50.768311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1db2be3e-e2b2-4618-9b1f-1b591e05ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.789970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2be3e-e2b2-4618-9b1f-1b591e05ede5 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:50.847043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d48a7c-87fa-443a-9b63-2fd98c2faa8f admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:50.853623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:50.854421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:50.854795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:50.855146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.857389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.882921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:50.921721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22309be0-1037-4cae-a063-b05af919cf8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:50.922426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22309be0-1037-4cae-a063-b05af919cf8f service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:50.922709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22309be0-1037-4cae-a063-b05af919cf8f service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:50.922990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22309be0-1037-4cae-a063-b05af919cf8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.929297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-22309be0-1037-4cae-a063-b05af919cf8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.940426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:50.942600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:50.955805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 897/1774] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:50.957471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:50.959429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:50.976323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6382421-00cc-40e6-bcdc-08c5adb337d6 service nova] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:50.980633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6e16b-2854-477f-a6e3-d5c9f2b082f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:50.981428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6e16b-2854-477f-a6e3-d5c9f2b082f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:50.981716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6e16b-2854-477f-a6e3-d5c9f2b082f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:50.982048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6e16b-2854-477f-a6e3-d5c9f2b082f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:50.985659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-25a6e16b-2854-477f-a6e3-d5c9f2b082f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:51.081134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6e16b-2854-477f-a6e3-d5c9f2b082f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:51.082579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6e16b-2854-477f-a6e3-d5c9f2b082f6 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:51.103901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 898/1775] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:51.107568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856dfe7-3f71-44e8-87b7-f7033fe22ed7 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:51.110096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856dfe7-3f71-44e8-87b7-f7033fe22ed7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:51.110096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856dfe7-3f71-44e8-87b7-f7033fe22ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:51.134753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22309be0-1037-4cae-a063-b05af919cf8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:51.136341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22309be0-1037-4cae-a063-b05af919cf8f service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:51.150111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-0856dfe7-3f71-44e8-87b7-f7033fe22ed7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:51.152413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0856dfe7-3f71-44e8-87b7-f7033fe22ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:51.183949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 878/1776] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:51.187671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330a359-0ba2-4d30-8ca9-6893d9f53c37 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:51.187957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330a359-0ba2-4d30-8ca9-6893d9f53c37 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:51.188323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330a359-0ba2-4d30-8ca9-6893d9f53c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:51.225671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:51.254669 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0330a359-0ba2-4d30-8ca9-6893d9f53c37 None None] MFA Rules not processed for user `23031b94dc7f47bf8272b266cbe18a8c`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:51.260575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0330a359-0ba2-4d30-8ca9-6893d9f53c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:51.439353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 899/1777] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:51.443958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10edad3-bea8-4fb5-8700-3dd250da16ad None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:51.445461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10edad3-bea8-4fb5-8700-3dd250da16ad None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:51.447590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10edad3-bea8-4fb5-8700-3dd250da16ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:51.478574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 879/1778] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:51.480642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330a359-0ba2-4d30-8ca9-6893d9f53c37 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:51.484145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0330a359-0ba2-4d30-8ca9-6893d9f53c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:51.504187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a10edad3-bea8-4fb5-8700-3dd250da16ad None None] MFA Rules not processed for user `9a67a5c4b7034d22b78331b907cc20bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:51.513855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a10edad3-bea8-4fb5-8700-3dd250da16ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:51.652834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330a359-0ba2-4d30-8ca9-6893d9f53c37 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:51.672450 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9cd55-2188-40c4-8a88-e5265779b09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:51.673170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9cd55-2188-40c4-8a88-e5265779b09c service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:51.673455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9cd55-2188-40c4-8a88-e5265779b09c service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:51.673735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9cd55-2188-40c4-8a88-e5265779b09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:51.686519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9cd55-2188-40c4-8a88-e5265779b09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:51.688225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9cd55-2188-40c4-8a88-e5265779b09c service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:51.705570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 880/1779] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:51.706807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9cd55-2188-40c4-8a88-e5265779b09c service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:51.722282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 900/1780] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:51.723864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d549de-2984-4544-9f09-bb7af27a0ea7 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:51.724785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d549de-2984-4544-9f09-bb7af27a0ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:51.724785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d549de-2984-4544-9f09-bb7af27a0ea7 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:51.726832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 901/1781] 10.222.0.52 () {54 vars in 912 bytes} [Tue Jun 21 15:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:53:51.730809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9cd55-2188-40c4-8a88-e5265779b09c service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:51.738195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087b4e5-7b16-4645-ba13-afa883697083 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:51.739722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087b4e5-7b16-4645-ba13-afa883697083 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:51.740661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:51.741005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087b4e5-7b16-4645-ba13-afa883697083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:51.743158 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:51.744006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:51.744959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:51.756543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:51.765436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:51.767458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:51.794295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 881/1782] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:51.844860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c087b4e5-7b16-4645-ba13-afa883697083 None None] MFA Rules not processed for user `6efa4bed025741d492158b5c6e45c7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:51.849124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c087b4e5-7b16-4645-ba13-afa883697083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:51.872882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 902/1783] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:53:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:51.948803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:51.950664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:52.001230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034b7c6-d40b-4436-9183-e217a229d49a service nova] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:52.022558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f82056-0c33-4dbd-8199-50c11585ccc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:52.023228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82056-0c33-4dbd-8199-50c11585ccc7 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:52.023513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82056-0c33-4dbd-8199-50c11585ccc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:52.023790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f82056-0c33-4dbd-8199-50c11585ccc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:52.053654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f82056-0c33-4dbd-8199-50c11585ccc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:52.056775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f82056-0c33-4dbd-8199-50c11585ccc7 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:52.081082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 882/1784] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:52.549062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087b4e5-7b16-4645-ba13-afa883697083 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:52.551375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c087b4e5-7b16-4645-ba13-afa883697083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:52.578054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087b4e5-7b16-4645-ba13-afa883697083 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:52.586763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:52.587658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:52.587940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:52.588216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] PATH_INFO: `/v3/projects/bc9f7f1f0a754ab9b2f6e46c842382e3` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:52.596005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc9f7f1f0a754ab9b2f6e46c842382e3)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:52.598129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:52.687436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6aca-de7b-4053-844c-08b4ec8ed27e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:52.687833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6aca-de7b-4053-844c-08b4ec8ed27e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:52.688476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e6aca-de7b-4053-844c-08b4ec8ed27e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:52.690188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 883/1785] 10.222.0.52 () {54 vars in 912 bytes} [Tue Jun 21 15:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 21 15:53:52.696355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b7ff-5162-4771-8a4b-e3187f70f2f6 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:52.697209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b7ff-5162-4771-8a4b-e3187f70f2f6 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:52.697500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8b7ff-5162-4771-8a4b-e3187f70f2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:52.722875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc9f7f1f0a754ab9b2f6e46c842382e3', 'request_id': 'req-2f239b67-9bb4-413b-b40c-9f035c46bd48'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:52.731929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-9db8b7ff-5162-4771-8a4b-e3187f70f2f6 None None] MFA Rules not processed for user `6efa4bed025741d492158b5c6e45c7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:52.736518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9db8b7ff-5162-4771-8a4b-e3187f70f2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:52.737279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc9f7f1f0a754ab9b2f6e46c842382e3', 'request_id': 'req-2f239b67-9bb4-413b-b40c-9f035c46bd48'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:52.754219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc9f7f1f0a754ab9b2f6e46c842382e3', 'request_id': 'req-2f239b67-9bb4-413b-b40c-9f035c46bd48'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:52.779253 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] The token cache is being invalidate because project bc9f7f1f0a754ab9b2f6e46c842382e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:52.779546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:52.782184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f239b67-9bb4-413b-b40c-9f035c46bd48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:52.785269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 903/1786] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:52 2022] DELETE /identity/v3/projects/bc9f7f1f0a754ab9b2f6e46c842382e3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:52.794937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6861e-eaf2-41bc-9583-d0e684e02bac None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:52.795251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6861e-eaf2-41bc-9583-d0e684e02bac None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:52.795505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6861e-eaf2-41bc-9583-d0e684e02bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:52.835393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-39b6861e-eaf2-41bc-9583-d0e684e02bac None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:52.838791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-39b6861e-eaf2-41bc-9583-d0e684e02bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:52.871852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 884/1787] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:53:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:52.983433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 904/1788] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:53:52.994228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372481dc-7fdc-4e6b-9a83-6c46a1a26e7a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:52.994549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372481dc-7fdc-4e6b-9a83-6c46a1a26e7a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:52.994858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372481dc-7fdc-4e6b-9a83-6c46a1a26e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.011973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6861e-eaf2-41bc-9583-d0e684e02bac None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:53.013492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-39b6861e-eaf2-41bc-9583-d0e684e02bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:53.030771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6861e-eaf2-41bc-9583-d0e684e02bac None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:53.035234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb7723-cb81-45ee-8a8e-e1dc6ebc2af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:53.036572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb7723-cb81-45ee-8a8e-e1dc6ebc2af0 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.036857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb7723-cb81-45ee-8a8e-e1dc6ebc2af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.037160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb7723-cb81-45ee-8a8e-e1dc6ebc2af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.042261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-372481dc-7fdc-4e6b-9a83-6c46a1a26e7a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:53.044121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-372481dc-7fdc-4e6b-9a83-6c46a1a26e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:53.048243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb7723-cb81-45ee-8a8e-e1dc6ebc2af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:53.054599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb7723-cb81-45ee-8a8e-e1dc6ebc2af0 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:53.065461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 885/1789] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:53.070032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372481dc-7fdc-4e6b-9a83-6c46a1a26e7a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:53.077453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 905/1790] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:53.086331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372481dc-7fdc-4e6b-9a83-6c46a1a26e7a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:53.092395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:53.093356 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.093901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.094403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.095570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:53:53.096089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:53.099739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:53.128970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 886/1791] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:53:53.130788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb7723-cb81-45ee-8a8e-e1dc6ebc2af0 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:53.145435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb7723-cb81-45ee-8a8e-e1dc6ebc2af0 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:53.154325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6f0de-ef7a-4085-b86e-c54d47987169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:53.154979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6f0de-ef7a-4085-b86e-c54d47987169 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.155384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6f0de-ef7a-4085-b86e-c54d47987169 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.155871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6f0de-ef7a-4085-b86e-c54d47987169 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.156595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6f0de-ef7a-4085-b86e-c54d47987169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:53.159080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6f0de-ef7a-4085-b86e-c54d47987169 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:53.203837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 906/1792] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:53 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:53:53.208527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:53.226507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ad42-31de-4144-88a0-bbedf3d10327 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:53.283857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:53.389474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58320f-95e8-4663-a0e7-3799ce19c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:53.390761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58320f-95e8-4663-a0e7-3799ce19c6ca admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.391673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58320f-95e8-4663-a0e7-3799ce19c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.392099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58320f-95e8-4663-a0e7-3799ce19c6ca admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.392999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58320f-95e8-4663-a0e7-3799ce19c6ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:53.396931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58320f-95e8-4663-a0e7-3799ce19c6ca admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:53.428428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 887/1793] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:53 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:53.432384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6f0de-ef7a-4085-b86e-c54d47987169 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:53.463075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6f0de-ef7a-4085-b86e-c54d47987169 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:53.472497 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:53.473580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.475215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.475763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.476745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:53.478660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:53.530733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 907/1794] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:53.535629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58320f-95e8-4663-a0e7-3799ce19c6ca admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:53.562434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58320f-95e8-4663-a0e7-3799ce19c6ca admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:53.589241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cb824-6a7c-404a-9344-00717c97ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:53.589966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cb824-6a7c-404a-9344-00717c97ad4e admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.590275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cb824-6a7c-404a-9344-00717c97ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.590533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cb824-6a7c-404a-9344-00717c97ad4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.596044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cb824-6a7c-404a-9344-00717c97ad4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:53.604069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cb824-6a7c-404a-9344-00717c97ad4e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:53.637096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:53.639249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:53.657989 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 888/1795] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:53.661218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cb824-6a7c-404a-9344-00717c97ad4e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:53.732040 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:53.737418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cb824-6a7c-404a-9344-00717c97ad4e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:53.758263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1a862-b618-4fc2-87ee-7094599b29c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:53.759206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a862-b618-4fc2-87ee-7094599b29c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.759494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a862-b618-4fc2-87ee-7094599b29c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.759775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a862-b618-4fc2-87ee-7094599b29c0 admin admin] PATH_INFO: `/v3/projects/dfa03d9101d444b98ba48382a08a25a7/users/4c451b8bad2e4fec853bc9cf6c785459/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.824525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b73dcc-7ea4-41ab-bba1-f40ee65c3f5d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:53.831150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19a902-5f99-46c0-8266-8a150f0c40a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:53.832059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19a902-5f99-46c0-8266-8a150f0c40a6 service placement] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.832437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19a902-5f99-46c0-8266-8a150f0c40a6 service placement] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.832831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19a902-5f99-46c0-8266-8a150f0c40a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.836404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2f19a902-5f99-46c0-8266-8a150f0c40a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:53.836576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1a862-b618-4fc2-87ee-7094599b29c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa03d9101d444b98ba48382a08a25a7, user_id=4c451b8bad2e4fec853bc9cf6c785459, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:53.846345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1a862-b618-4fc2-87ee-7094599b29c0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:53.851456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19a902-5f99-46c0-8266-8a150f0c40a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:53.853338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19a902-5f99-46c0-8266-8a150f0c40a6 service placement] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:53.872824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 908/1796] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:53:53.889349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 889/1797] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:53 2022] PUT /identity/v3/projects/dfa03d9101d444b98ba48382a08a25a7/users/4c451b8bad2e4fec853bc9cf6c785459/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:53.903643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23291875-1ad0-4daa-bf3e-19b8cd5d712f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:53.903931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23291875-1ad0-4daa-bf3e-19b8cd5d712f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:53.904560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23291875-1ad0-4daa-bf3e-19b8cd5d712f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:53.978939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-23291875-1ad0-4daa-bf3e-19b8cd5d712f None None] MFA Rules not processed for user `4c451b8bad2e4fec853bc9cf6c785459`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:53.982985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-23291875-1ad0-4daa-bf3e-19b8cd5d712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:54.182989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 909/1798] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:54.186447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1a862-b618-4fc2-87ee-7094599b29c0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:54.485926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1a862-b618-4fc2-87ee-7094599b29c0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:54.496070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23291875-1ad0-4daa-bf3e-19b8cd5d712f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:54.500227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-23291875-1ad0-4daa-bf3e-19b8cd5d712f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:54.501299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910b44-1eec-43ea-b231-2d6853984c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:54.502260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910b44-1eec-43ea-b231-2d6853984c96 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:54.508770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910b44-1eec-43ea-b231-2d6853984c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:54.510189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910b44-1eec-43ea-b231-2d6853984c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:54.511167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b910b44-1eec-43ea-b231-2d6853984c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:54.514912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b910b44-1eec-43ea-b231-2d6853984c96 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:54.529660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23291875-1ad0-4daa-bf3e-19b8cd5d712f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:54.541531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:54.542461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:54.545100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:54.545430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] PATH_INFO: `/v3/users/2e36d610bb1447b0a06e6764780357b3` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:54.547539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e36d610bb1447b0a06e6764780357b3)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:54.553467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:54.591998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 890/1799] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:53:54 2022] POST /identity/v3/projects => generated 371 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:54.592944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910b44-1eec-43ea-b231-2d6853984c96 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:54.702933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e36d610bb1447b0a06e6764780357b3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:54.727119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e36d610bb1447b0a06e6764780357b3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:54.744037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e36d610bb1447b0a06e6764780357b3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:54.744748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e36d610bb1447b0a06e6764780357b3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:54.775379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5cbde524-50d0-4bc1-9efa-beff043700cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e36d610bb1447b0a06e6764780357b3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:54.791016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:54.801172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 910/1800] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:54 2022] DELETE /identity/v3/users/2e36d610bb1447b0a06e6764780357b3 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:54.806094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375c6ab-dc01-43cc-89e6-16a260ed5a1d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:54.806282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375c6ab-dc01-43cc-89e6-16a260ed5a1d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:54.808907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910b44-1eec-43ea-b231-2d6853984c96 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:54.809723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375c6ab-dc01-43cc-89e6-16a260ed5a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:54.850627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e375c6ab-dc01-43cc-89e6-16a260ed5a1d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:54.851919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e375c6ab-dc01-43cc-89e6-16a260ed5a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:54.874098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:54.994419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:54.995192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:54.995481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:54.995760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:54.996597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:55.000801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:55.057261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 891/1801] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:53:54 2022] POST /identity/v3/users => generated 292 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:53:55.068036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:55.071746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:55.079714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 911/1802] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:55.100656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375c6ab-dc01-43cc-89e6-16a260ed5a1d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:55.102699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e375c6ab-dc01-43cc-89e6-16a260ed5a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:55.103825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569494d2-9737-490f-8822-d92c444e548b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:55.116929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b4bbe-71c1-4ccf-b2ac-ccee5fccd2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:55.117797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b4bbe-71c1-4ccf-b2ac-ccee5fccd2cd admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:55.119151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b4bbe-71c1-4ccf-b2ac-ccee5fccd2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:55.119440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b4bbe-71c1-4ccf-b2ac-ccee5fccd2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:55.122992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b4bbe-71c1-4ccf-b2ac-ccee5fccd2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:55.129552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b4bbe-71c1-4ccf-b2ac-ccee5fccd2cd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:55.169830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 892/1803] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:55.173604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b4bbe-71c1-4ccf-b2ac-ccee5fccd2cd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:55.226722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b4bbe-71c1-4ccf-b2ac-ccee5fccd2cd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:55.252199 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b2a88-eb3a-43f2-98f8-6a14c53758b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:55.254705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b2a88-eb3a-43f2-98f8-6a14c53758b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:55.254994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b2a88-eb3a-43f2-98f8-6a14c53758b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:55.255478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b2a88-eb3a-43f2-98f8-6a14c53758b7 admin admin] PATH_INFO: `/v3/projects/748e57b8117f430ebc40c6c4ed784f2e/users/97c8cdb6a84c4a389b0b11f16d2bd112/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:55.267730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375c6ab-dc01-43cc-89e6-16a260ed5a1d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:55.272958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c0a62-af3b-4e82-9e46-0cffe45ef42b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:55.274025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0a62-af3b-4e82-9e46-0cffe45ef42b service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:55.274534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0a62-af3b-4e82-9e46-0cffe45ef42b service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:55.275274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0a62-af3b-4e82-9e46-0cffe45ef42b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:55.291994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c0a62-af3b-4e82-9e46-0cffe45ef42b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:55.295853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c0a62-af3b-4e82-9e46-0cffe45ef42b service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:55.342808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b2a88-eb3a-43f2-98f8-6a14c53758b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=748e57b8117f430ebc40c6c4ed784f2e, user_id=97c8cdb6a84c4a389b0b11f16d2bd112, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:55.350293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 912/1804] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:53:55.363971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b2a88-eb3a-43f2-98f8-6a14c53758b7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:55.421697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 893/1805] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:55 2022] PUT /identity/v3/projects/748e57b8117f430ebc40c6c4ed784f2e/users/97c8cdb6a84c4a389b0b11f16d2bd112/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:55.423916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c0a62-af3b-4e82-9e46-0cffe45ef42b service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:55.468563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c0a62-af3b-4e82-9e46-0cffe45ef42b service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:55.778709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b9a45-5cae-4b34-8aca-37fe0da284bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:55.779702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9a45-5cae-4b34-8aca-37fe0da284bb admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:55.780300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9a45-5cae-4b34-8aca-37fe0da284bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:55.780646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9a45-5cae-4b34-8aca-37fe0da284bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:55.782162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b9a45-5cae-4b34-8aca-37fe0da284bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:55.783956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b9a45-5cae-4b34-8aca-37fe0da284bb admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:55.813057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 913/1806] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:53:55.820812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b2a88-eb3a-43f2-98f8-6a14c53758b7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:55.842657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b2a88-eb3a-43f2-98f8-6a14c53758b7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:55.852897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51855e-6ae8-4227-a75e-1159ffb43290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:55.854266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51855e-6ae8-4227-a75e-1159ffb43290 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:55.854842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51855e-6ae8-4227-a75e-1159ffb43290 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:55.855136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51855e-6ae8-4227-a75e-1159ffb43290 admin admin] PATH_INFO: `/v3/projects/748e57b8117f430ebc40c6c4ed784f2e/users/97c8cdb6a84c4a389b0b11f16d2bd112/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:55.857892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51855e-6ae8-4227-a75e-1159ffb43290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=748e57b8117f430ebc40c6c4ed784f2e, user_id=97c8cdb6a84c4a389b0b11f16d2bd112, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:55.861196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51855e-6ae8-4227-a75e-1159ffb43290 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:55.875470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 894/1807] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:53:55 2022] PUT /identity/v3/projects/748e57b8117f430ebc40c6c4ed784f2e/users/97c8cdb6a84c4a389b0b11f16d2bd112/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:55.886451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531251f-b332-43c3-9b43-1961e130c68a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:55.887266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531251f-b332-43c3-9b43-1961e130c68a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:55.888215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531251f-b332-43c3-9b43-1961e130c68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:56.025485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b531251f-b332-43c3-9b43-1961e130c68a None None] MFA Rules not processed for user `97c8cdb6a84c4a389b0b11f16d2bd112`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:56.032138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b531251f-b332-43c3-9b43-1961e130c68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:56.169985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:56.199351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace9b19-b69a-4a5e-a18f-d2c86da956c7 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:56.199649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace9b19-b69a-4a5e-a18f-d2c86da956c7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:56.199926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace9b19-b69a-4a5e-a18f-d2c86da956c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:56.256100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 914/1808] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:55 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:56.269679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531251f-b332-43c3-9b43-1961e130c68a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:56.271683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b531251f-b332-43c3-9b43-1961e130c68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:56.276907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0ace9b19-b69a-4a5e-a18f-d2c86da956c7 None None] MFA Rules not processed for user `75407a21b1fd46e8ba10540fc5a4c30a`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:56.281344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0ace9b19-b69a-4a5e-a18f-d2c86da956c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:56.469349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 895/1809] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:56.553250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531251f-b332-43c3-9b43-1961e130c68a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:56.565150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f1b47-0e1c-4045-89c3-9a19875f55d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:56.566339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f1b47-0e1c-4045-89c3-9a19875f55d0 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:56.567038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f1b47-0e1c-4045-89c3-9a19875f55d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:56.567385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f1b47-0e1c-4045-89c3-9a19875f55d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:56.614252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f1b47-0e1c-4045-89c3-9a19875f55d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:56.616344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f1b47-0e1c-4045-89c3-9a19875f55d0 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:56.660307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 915/1810] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:56 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:56.700998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace9b19-b69a-4a5e-a18f-d2c86da956c7 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:56.702591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0ace9b19-b69a-4a5e-a18f-d2c86da956c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:56.891231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:56.957624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace9b19-b69a-4a5e-a18f-d2c86da956c7 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:56.972526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:56.977120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:56.979895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:56.982593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] PATH_INFO: `/v3/projects/20fac0197749438b8665b82cd70b6890` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:56.993512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] RBAC: Authorizing `identity:delete_project(project_id=20fac0197749438b8665b82cd70b6890)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:57.000064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:57.188276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20fac0197749438b8665b82cd70b6890', 'request_id': 'req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.207039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20fac0197749438b8665b82cd70b6890', 'request_id': 'req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.224591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20fac0197749438b8665b82cd70b6890', 'request_id': 'req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.261939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] The token cache is being invalidate because project 20fac0197749438b8665b82cd70b6890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:53:57.262315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.264040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.268832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 896/1811] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:56 2022] DELETE /identity/v3/projects/20fac0197749438b8665b82cd70b6890 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:57.272741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f1b47-0e1c-4045-89c3-9a19875f55d0 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:57.288327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f1b47-0e1c-4045-89c3-9a19875f55d0 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:57.333729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:57.335616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:57.465414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:57.467713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:57.473024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:57.473416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] PATH_INFO: `/v3/users/8be157c0ba61493eade91da1d0ab2688` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:57.480028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be157c0ba61493eade91da1d0ab2688)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:57.489697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:57.549406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b6680-414f-44cc-a681-9c9d3e5bb23a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:57.556641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:57.557847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:57.558371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:57.558998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] PATH_INFO: `/v3/users/b0f67d2032284d0f93984ab05f2233d7` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:57.560709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f67d2032284d0f93984ab05f2233d7)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:57.578253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:57.659080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be157c0ba61493eade91da1d0ab2688'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.699693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be157c0ba61493eade91da1d0ab2688'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.706753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f67d2032284d0f93984ab05f2233d7'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.719712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be157c0ba61493eade91da1d0ab2688'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.720569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be157c0ba61493eade91da1d0ab2688'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.745545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df8c6af1-073b-42e1-abe6-fba885862f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be157c0ba61493eade91da1d0ab2688'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.749606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:57.765050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 916/1812] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:57 2022] DELETE /identity/v3/users/8be157c0ba61493eade91da1d0ab2688 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:53:57.769083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f67d2032284d0f93984ab05f2233d7'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.784888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f67d2032284d0f93984ab05f2233d7'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.801232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f67d2032284d0f93984ab05f2233d7'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.805227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f67d2032284d0f93984ab05f2233d7'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:57.807258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:57.814240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 897/1813] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:57 2022] DELETE /identity/v3/users/b0f67d2032284d0f93984ab05f2233d7 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:53:57.819305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49162da-a4cd-4b58-b544-f6b2eedd7db5 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:57.820030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49162da-a4cd-4b58-b544-f6b2eedd7db5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:57.820504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49162da-a4cd-4b58-b544-f6b2eedd7db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:57.850658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f49162da-a4cd-4b58-b544-f6b2eedd7db5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:57.852743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f49162da-a4cd-4b58-b544-f6b2eedd7db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:57.959896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:57.961091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:57.988501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 917/1814] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:58.069284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49162da-a4cd-4b58-b544-f6b2eedd7db5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:58.071529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f49162da-a4cd-4b58-b544-f6b2eedd7db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:58.238427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d43086-2434-48c6-8486-f0e11d5fa3ac admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:58.246310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:58.247527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:58.247952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:58.248365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] PATH_INFO: `/v3/users/4c451b8bad2e4fec853bc9cf6c785459` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:58.250653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c451b8bad2e4fec853bc9cf6c785459)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:58.253514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:58.314629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49162da-a4cd-4b58-b544-f6b2eedd7db5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:58.335910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05574086-5931-423d-ad4e-2836bec5342b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:58.336793 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05574086-5931-423d-ad4e-2836bec5342b service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:58.337735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05574086-5931-423d-ad4e-2836bec5342b service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:58.338394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05574086-5931-423d-ad4e-2836bec5342b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:58.358061 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05574086-5931-423d-ad4e-2836bec5342b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:58.361175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05574086-5931-423d-ad4e-2836bec5342b service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:58.374238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 918/1815] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:58.410158 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c451b8bad2e4fec853bc9cf6c785459'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:58.462239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c451b8bad2e4fec853bc9cf6c785459'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:58.487827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c451b8bad2e4fec853bc9cf6c785459'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:58.509247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c451b8bad2e4fec853bc9cf6c785459'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:58.512697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-04842615-585c-4312-8e11-e404012e8f4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c451b8bad2e4fec853bc9cf6c785459'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:53:58.531859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 898/1816] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:53:57 2022] DELETE /identity/v3/users/4c451b8bad2e4fec853bc9cf6c785459 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:53:58.537199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9c6ed-31ff-492d-8714-a44bec1f17f7 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:58.537805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9c6ed-31ff-492d-8714-a44bec1f17f7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:58.538166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9c6ed-31ff-492d-8714-a44bec1f17f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:58.604973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-cda9c6ed-31ff-492d-8714-a44bec1f17f7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:58.611230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cda9c6ed-31ff-492d-8714-a44bec1f17f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:58.611720 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4280f4-f856-458a-8aee-c85897b3a057 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:58.616635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4280f4-f856-458a-8aee-c85897b3a057 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:58.616893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4280f4-f856-458a-8aee-c85897b3a057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:58.653254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-bd4280f4-f856-458a-8aee-c85897b3a057 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:53:58.654915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bd4280f4-f856-458a-8aee-c85897b3a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:58.788918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 919/1817] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:53:58.803663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 899/1818] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:53:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:53:58.825678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9c6ed-31ff-492d-8714-a44bec1f17f7 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:58.827371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cda9c6ed-31ff-492d-8714-a44bec1f17f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:58.998693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:59.025249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9c6ed-31ff-492d-8714-a44bec1f17f7 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:59.034242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:59.035000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:59.035274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:59.035574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:59.063158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:59.064561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:59.097841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 920/1819] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:53:59.112036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4280f4-f856-458a-8aee-c85897b3a057 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:59.114920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bd4280f4-f856-458a-8aee-c85897b3a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:59.184110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:53:59.277382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4280f4-f856-458a-8aee-c85897b3a057 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:59.283535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98b367-639f-427b-85f1-0bc251488385 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:59.284333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b367-639f-427b-85f1-0bc251488385 service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:59.285270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b367-639f-427b-85f1-0bc251488385 service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:59.286272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98b367-639f-427b-85f1-0bc251488385 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:59.297872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98b367-639f-427b-85f1-0bc251488385 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:59.300013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98b367-639f-427b-85f1-0bc251488385 service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:53:59.314414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 900/1820] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:53:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:53:59.658739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:53:59.660456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:53:59.685667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422eb82-ca17-4806-bad1-391c7f84e301 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:53:59.707573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:53:59.712602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:53:59.714203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:53:59.716622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] PATH_INFO: `/v3/projects/dcf51e917d4f448fa7ed5e1b77cb4f4e` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:53:59.720535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf51e917d4f448fa7ed5e1b77cb4f4e)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:53:59.728105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:00.023812 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf51e917d4f448fa7ed5e1b77cb4f4e', 'request_id': 'req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:00.057298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf51e917d4f448fa7ed5e1b77cb4f4e', 'request_id': 'req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:00.089230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf51e917d4f448fa7ed5e1b77cb4f4e', 'request_id': 'req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:00.151632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] The token cache is being invalidate because project dcf51e917d4f448fa7ed5e1b77cb4f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:00.151985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:00.154760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:00.158228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 921/1821] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:53:59 2022] DELETE /identity/v3/projects/dcf51e917d4f448fa7ed5e1b77cb4f4e => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:00.164878 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77e459-87fc-4bdb-aff9-dda668dbdb4f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:00.165395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77e459-87fc-4bdb-aff9-dda668dbdb4f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:00.165943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77e459-87fc-4bdb-aff9-dda668dbdb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:00.242030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ee77e459-87fc-4bdb-aff9-dda668dbdb4f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:00.243304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ee77e459-87fc-4bdb-aff9-dda668dbdb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:00.358070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:00.359816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:00.621691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 901/1822] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:00.631344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd5db-814b-4251-a21d-79061fde7ca4 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:00.632049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd5db-814b-4251-a21d-79061fde7ca4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:00.632816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cd5db-814b-4251-a21d-79061fde7ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:00.677326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-902cd5db-814b-4251-a21d-79061fde7ca4 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:00.682980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-902cd5db-814b-4251-a21d-79061fde7ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:00.708967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e392d6-2ecd-47e0-a505-d2feba2e8f6f admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:00.724056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 902/1823] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:00.729689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cd5db-814b-4251-a21d-79061fde7ca4 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:00.750073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:00.751045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:00.751335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:00.751623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] PATH_INFO: `/v3/users/dd3b26c356e34584a38eceea2ec1f57f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:00.762458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3b26c356e34584a38eceea2ec1f57f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:00.768850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:00.786477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cd5db-814b-4251-a21d-79061fde7ca4 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:00.805581 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:00.818442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:00.819837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:00.824439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:00.825408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:00.826655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:00.841414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:00.906549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 903/1824] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:54:00.907497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:00.912291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:00.966078 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3b26c356e34584a38eceea2ec1f57f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:00.967582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f37c-8dea-425f-b5dd-24be06ddbef1 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:00.987211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:00.988193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:00.989488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:00.990042 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] PATH_INFO: `/v3/projects/b8ded3e9f6d24612a29985564f1ea7eb` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:00.994981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ded3e9f6d24612a29985564f1ea7eb)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:01.000606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3b26c356e34584a38eceea2ec1f57f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.005265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:01.024579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3b26c356e34584a38eceea2ec1f57f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.025551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3b26c356e34584a38eceea2ec1f57f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.028323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:01.043265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3b26c356e34584a38eceea2ec1f57f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.054238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 922/1825] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:00 2022] DELETE /identity/v3/users/dd3b26c356e34584a38eceea2ec1f57f => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:01.060830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:01.062837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:01.192051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ded3e9f6d24612a29985564f1ea7eb', 'request_id': 'req-83f1272a-db39-4987-a278-65e7b84e2cc1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.207681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ded3e9f6d24612a29985564f1ea7eb', 'request_id': 'req-83f1272a-db39-4987-a278-65e7b84e2cc1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.219240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ded3e9f6d24612a29985564f1ea7eb', 'request_id': 'req-83f1272a-db39-4987-a278-65e7b84e2cc1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.254722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ed063-da17-40b4-8881-cba0b518e494 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:01.264761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] The token cache is being invalidate because project b8ded3e9f6d24612a29985564f1ea7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:01.266812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.269614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-83f1272a-db39-4987-a278-65e7b84e2cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:01.274705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 904/1826] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:00 2022] DELETE /identity/v3/projects/b8ded3e9f6d24612a29985564f1ea7eb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:01.279110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede3c40-913c-4216-b1f4-a36fc67cad14 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:01.279753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede3c40-913c-4216-b1f4-a36fc67cad14 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:01.280335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede3c40-913c-4216-b1f4-a36fc67cad14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:01.338174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1ede3c40-913c-4216-b1f4-a36fc67cad14 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:01.341855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1ede3c40-913c-4216-b1f4-a36fc67cad14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:01.373136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:01.423424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:01.424259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:01.427531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:01.427973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:01.429991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:01.435138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:01.542182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 923/1827] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:01 2022] POST /identity/v3/projects => generated 385 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:01.557494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:01.560723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:01.577005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 905/1828] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:01.577905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede3c40-913c-4216-b1f4-a36fc67cad14 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:01.581044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1ede3c40-913c-4216-b1f4-a36fc67cad14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:01.621441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede3c40-913c-4216-b1f4-a36fc67cad14 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:01.785322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf1b1-fe41-450f-a45a-b0c664b53f5d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:01.793855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:01.795573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:01.796019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:01.796459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] PATH_INFO: `/v3/users/7f28af369cac4351b9350bd8317fa2a3` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:01.800471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f28af369cac4351b9350bd8317fa2a3)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:01.805122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:01.832522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:01.834617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:01.835019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:01.835990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:01.836829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:01.843975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:01.884031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:01.919191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 906/1829] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:01 2022] POST /identity/v3/users => generated 293 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:01.921835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:01.931732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:01.974415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f28af369cac4351b9350bd8317fa2a3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.007715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f28af369cac4351b9350bd8317fa2a3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.019694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f28af369cac4351b9350bd8317fa2a3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.021270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f28af369cac4351b9350bd8317fa2a3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.032333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f28af369cac4351b9350bd8317fa2a3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.044959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 924/1830] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:01 2022] DELETE /identity/v3/users/7f28af369cac4351b9350bd8317fa2a3 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:02.044959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:02.056108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:02.113399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71f8c7-db84-4875-bc37-1fe3e77d2fc4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:02.290098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:02.290907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:02.291239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:02.291888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:02.304218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:02.336198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c33f0-ec23-4886-95c6-bb7023822b3c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:02.352943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:02.355121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:02.356364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:02.358195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:02.359965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:02.367444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] PATH_INFO: `/v3/projects/dfa03d9101d444b98ba48382a08a25a7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:02.367444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa03d9101d444b98ba48382a08a25a7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:02.369857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:02.400066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 907/1831] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:02.407096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:02.414913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:02.474187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:02.486384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f5d5f-2f53-4057-8f15-c7c759ac4530 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:02.496012 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa03d9101d444b98ba48382a08a25a7', 'request_id': 'req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.508767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa03d9101d444b98ba48382a08a25a7', 'request_id': 'req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.518164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa03d9101d444b98ba48382a08a25a7', 'request_id': 'req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.540996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] The token cache is being invalidate because project dfa03d9101d444b98ba48382a08a25a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:02.542204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.543176 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.544966 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 925/1832] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:02 2022] DELETE /identity/v3/projects/dfa03d9101d444b98ba48382a08a25a7 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:02.550873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:02.553331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:02.615427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:02.672208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:02.673136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:02.686023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:02.686805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:02.687738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:02.689791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:02.714616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 908/1833] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:02.714840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5a58c-771a-43dc-8476-6fe4faa5f3e6 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:02.716567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:02.720891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:02.728154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:02.729343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:02.729726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:02.731295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] PATH_INFO: `/v3/users/97c8cdb6a84c4a389b0b11f16d2bd112` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:02.735615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c8cdb6a84c4a389b0b11f16d2bd112)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:02.742147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:02.755529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e0841-5a81-4e48-86b6-35a423463ff8 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:02.768788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e7282-7302-42c5-9871-da379500cec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:02.769906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7282-7302-42c5-9871-da379500cec1 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:02.770785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7282-7302-42c5-9871-da379500cec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:02.770785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e7282-7302-42c5-9871-da379500cec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:02.780246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e7282-7302-42c5-9871-da379500cec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:02.785274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e7282-7302-42c5-9871-da379500cec1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:02.813820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 909/1834] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:02.821542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e7282-7302-42c5-9871-da379500cec1 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:02.885619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e7282-7302-42c5-9871-da379500cec1 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:02.904903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c8cdb6a84c4a389b0b11f16d2bd112'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.905554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc254d7-dcb4-4b4d-a843-d230152eb5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:02.906872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc254d7-dcb4-4b4d-a843-d230152eb5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:02.908096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc254d7-dcb4-4b4d-a843-d230152eb5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:02.908684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc254d7-dcb4-4b4d-a843-d230152eb5cf admin admin] PATH_INFO: `/v3/projects/c749343f39de4722999ca748ad05be32/users/5dd369eba6a64bba8651983ed9e28d78/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:02.962292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c8cdb6a84c4a389b0b11f16d2bd112'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.974547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc254d7-dcb4-4b4d-a843-d230152eb5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c749343f39de4722999ca748ad05be32, user_id=5dd369eba6a64bba8651983ed9e28d78, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:02.978520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c8cdb6a84c4a389b0b11f16d2bd112'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.980297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c8cdb6a84c4a389b0b11f16d2bd112'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:02.994985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc254d7-dcb4-4b4d-a843-d230152eb5cf admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:03.019246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b0cb4d39-62c0-4cc7-ba44-0f3e41f4ae7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c8cdb6a84c4a389b0b11f16d2bd112'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:03.041429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 910/1835] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:02 2022] PUT /identity/v3/projects/c749343f39de4722999ca748ad05be32/users/5dd369eba6a64bba8651983ed9e28d78/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:03.044594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3333f0-02b2-452d-9308-006d46728d59 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:03.044594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3333f0-02b2-452d-9308-006d46728d59 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:03.051122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 926/1836] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:02 2022] DELETE /identity/v3/users/97c8cdb6a84c4a389b0b11f16d2bd112 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:03.052051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3333f0-02b2-452d-9308-006d46728d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:03.055303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4f55-5551-4d95-88e5-da8887854293 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:03.056280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4f55-5551-4d95-88e5-da8887854293 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:03.057296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4f55-5551-4d95-88e5-da8887854293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:03.134176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7c3333f0-02b2-452d-9308-006d46728d59 None None] MFA Rules not processed for user `6efa4bed025741d492158b5c6e45c7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:03.136610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7c3333f0-02b2-452d-9308-006d46728d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:03.166898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a50f4f55-5551-4d95-88e5-da8887854293 None None] MFA Rules not processed for user `5dd369eba6a64bba8651983ed9e28d78`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:03.168178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a50f4f55-5551-4d95-88e5-da8887854293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:03.294354 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 911/1837] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Jun 21 15:54:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:03.395888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 927/1838] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:03.400609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3333f0-02b2-452d-9308-006d46728d59 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:03.402471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7c3333f0-02b2-452d-9308-006d46728d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:03.419822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f4f55-5551-4d95-88e5-da8887854293 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:03.421846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a50f4f55-5551-4d95-88e5-da8887854293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:03.454550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f4f55-5551-4d95-88e5-da8887854293 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:03.460919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718a662-0d39-4767-a83d-f8c5b79fc8e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:03.461806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718a662-0d39-4767-a83d-f8c5b79fc8e3 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:03.461927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718a662-0d39-4767-a83d-f8c5b79fc8e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:03.462175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718a662-0d39-4767-a83d-f8c5b79fc8e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:03.480487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718a662-0d39-4767-a83d-f8c5b79fc8e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:03.481916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718a662-0d39-4767-a83d-f8c5b79fc8e3 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:03.531050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 928/1839] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:03.564814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3333f0-02b2-452d-9308-006d46728d59 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:03.582946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e7332-93f5-4091-9ed0-5a75431ad2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:03.585235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e7332-93f5-4091-9ed0-5a75431ad2a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:03.585855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e7332-93f5-4091-9ed0-5a75431ad2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:03.586408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e7332-93f5-4091-9ed0-5a75431ad2a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:03.587598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e7332-93f5-4091-9ed0-5a75431ad2a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:03.590772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e7332-93f5-4091-9ed0-5a75431ad2a0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:03.625618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:03.639918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 912/1840] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:03 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:03.643512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718a662-0d39-4767-a83d-f8c5b79fc8e3 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:03.660472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718a662-0d39-4767-a83d-f8c5b79fc8e3 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:03.721112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:03.782032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf9155-3a90-4b4a-b931-2b03426fac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:03.783045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf9155-3a90-4b4a-b931-2b03426fac8c admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:03.783691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf9155-3a90-4b4a-b931-2b03426fac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:03.783691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf9155-3a90-4b4a-b931-2b03426fac8c admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:03.784164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf9155-3a90-4b4a-b931-2b03426fac8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:03.786520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf9155-3a90-4b4a-b931-2b03426fac8c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:03.813419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 929/1841] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:03 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:03.817829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e7332-93f5-4091-9ed0-5a75431ad2a0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:03.843589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e7332-93f5-4091-9ed0-5a75431ad2a0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:03.857258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c002f-3915-474d-9265-e3effa833186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:03.859817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c002f-3915-474d-9265-e3effa833186 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:03.860099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c002f-3915-474d-9265-e3effa833186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:03.861584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c002f-3915-474d-9265-e3effa833186 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:03.864088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c002f-3915-474d-9265-e3effa833186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:03.869346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c002f-3915-474d-9265-e3effa833186 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:03.909863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 913/1842] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:03.910967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf9155-3a90-4b4a-b931-2b03426fac8c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:03.928185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf9155-3a90-4b4a-b931-2b03426fac8c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:03.942815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b308d8-8f7c-446b-b7aa-b1c0aedfab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:03.942815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b308d8-8f7c-446b-b7aa-b1c0aedfab3e admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:03.942815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b308d8-8f7c-446b-b7aa-b1c0aedfab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:03.942815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b308d8-8f7c-446b-b7aa-b1c0aedfab3e admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:03.943427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b308d8-8f7c-446b-b7aa-b1c0aedfab3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:03.943427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b308d8-8f7c-446b-b7aa-b1c0aedfab3e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:03.979191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c002f-3915-474d-9265-e3effa833186 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:03.979636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 930/1843] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:04.030487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c002f-3915-474d-9265-e3effa833186 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:04.043436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:04.044843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:04.045419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:04.046055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] PATH_INFO: `/v3/projects/6692fa0997f8426eadb3fc5302626234/users/92fa886675704f708a3ca9f2dbb549b4/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:04.128138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6692fa0997f8426eadb3fc5302626234, user_id=92fa886675704f708a3ca9f2dbb549b4, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:04.159196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:04.224215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 914/1844] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:03 2022] PUT /identity/v3/projects/6692fa0997f8426eadb3fc5302626234/users/92fa886675704f708a3ca9f2dbb549b4/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:04.244611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90d6da-bc41-4aa8-91ff-45ae427e4a79 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:04.244896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90d6da-bc41-4aa8-91ff-45ae427e4a79 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:04.245160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90d6da-bc41-4aa8-91ff-45ae427e4a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:04.322850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2d90d6da-bc41-4aa8-91ff-45ae427e4a79 None None] MFA Rules not processed for user `92fa886675704f708a3ca9f2dbb549b4`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:04.326272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2d90d6da-bc41-4aa8-91ff-45ae427e4a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:04.368990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:04.371980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:04.546556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 931/1845] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:04.551685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90d6da-bc41-4aa8-91ff-45ae427e4a79 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:04.601363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1a3e1-6d89-4198-8186-8121447cfc17 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:04.614705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:04.616267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:04.616767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:04.617250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] PATH_INFO: `/v3/projects/5dd4fcc787e844d9997c1ae283584e09` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:04.620197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd4fcc787e844d9997c1ae283584e09)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:04.622437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:04.714173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90d6da-bc41-4aa8-91ff-45ae427e4a79 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:04.716176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd4fcc787e844d9997c1ae283584e09', 'request_id': 'req-67700ccc-8141-45f0-8c6b-dfd66d3480a0'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:04.726839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd4fcc787e844d9997c1ae283584e09', 'request_id': 'req-67700ccc-8141-45f0-8c6b-dfd66d3480a0'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:04.745872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd4fcc787e844d9997c1ae283584e09', 'request_id': 'req-67700ccc-8141-45f0-8c6b-dfd66d3480a0'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:04.772272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] The token cache is being invalidate because project 5dd4fcc787e844d9997c1ae283584e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:04.772604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:04.774296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:04.781318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 915/1846] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:04 2022] DELETE /identity/v3/projects/5dd4fcc787e844d9997c1ae283584e09 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:04.782400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:04.784461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:04.863265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:04.916046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:04.933025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67700ccc-8141-45f0-8c6b-dfd66d3480a0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:04.936545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:04.937634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:04.937957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:04.944963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:04.947198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:04.947349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:04.949090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:04.949490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:04.950885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:04.951447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] PATH_INFO: `/v3/users/4a653b37ea3a4270816a9993bd2e94d3` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:04.953358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a653b37ea3a4270816a9993bd2e94d3)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:04.955824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:05.037828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 932/1847] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:04 2022] POST /identity/v3/projects => generated 385 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:54:05.041615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:05.044334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:05.056280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a653b37ea3a4270816a9993bd2e94d3'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.092390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a653b37ea3a4270816a9993bd2e94d3'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.116724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a653b37ea3a4270816a9993bd2e94d3'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.191825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a653b37ea3a4270816a9993bd2e94d3'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.192732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a653b37ea3a4270816a9993bd2e94d3'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.194248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06ad9e-46ea-491e-8341-cef73d490c76 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:05.201178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:05.202100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:05.202535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:05.203017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:05.204524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:05.207234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:05.230932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 916/1848] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:04 2022] DELETE /identity/v3/users/4a653b37ea3a4270816a9993bd2e94d3 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:05.240978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:05.247562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:05.265293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 933/1849] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:05 2022] POST /identity/v3/users => generated 299 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:05.266637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:05.268535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:05.287901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e80eb-021d-44a5-9114-46d6ecb7dd2e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:05.368295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9ea9f-0ace-45b2-933d-a341f369e3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:05.369008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9ea9f-0ace-45b2-933d-a341f369e3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:05.369341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9ea9f-0ace-45b2-933d-a341f369e3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:05.369625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa9ea9f-0ace-45b2-933d-a341f369e3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:05.370411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9ea9f-0ace-45b2-933d-a341f369e3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:05.372349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa9ea9f-0ace-45b2-933d-a341f369e3d2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:05.401414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 934/1850] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:05.402813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9ea9f-0ace-45b2-933d-a341f369e3d2 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:05.432521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa9ea9f-0ace-45b2-933d-a341f369e3d2 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:05.454663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857b4bc-f25f-45a3-8028-adbb9d2bbc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:05.456146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857b4bc-f25f-45a3-8028-adbb9d2bbc02 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:05.456578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857b4bc-f25f-45a3-8028-adbb9d2bbc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:05.457011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857b4bc-f25f-45a3-8028-adbb9d2bbc02 admin admin] PATH_INFO: `/v3/projects/1be5c03fb3ec45558b9c70c24618247d/users/29ef1ef69cbe481c949431154655eecd/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:05.487420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b10da-0b4d-45f4-ad99-68398ddfc0e0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:05.517206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857b4bc-f25f-45a3-8028-adbb9d2bbc02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be5c03fb3ec45558b9c70c24618247d, user_id=29ef1ef69cbe481c949431154655eecd, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:05.520437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:05.520625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857b4bc-f25f-45a3-8028-adbb9d2bbc02 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:05.521989 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:05.522270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:05.522925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] PATH_INFO: `/v3/projects/748e57b8117f430ebc40c6c4ed784f2e` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:05.524000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=748e57b8117f430ebc40c6c4ed784f2e)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:05.532978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:05.553908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 935/1851] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:05 2022] PUT /identity/v3/projects/1be5c03fb3ec45558b9c70c24618247d/users/29ef1ef69cbe481c949431154655eecd/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:05.559084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857b4bc-f25f-45a3-8028-adbb9d2bbc02 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:05.673346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '748e57b8117f430ebc40c6c4ed784f2e', 'request_id': 'req-895b23be-3865-460a-80b9-adf3c3f7ccdf'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.675891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:05.689585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '748e57b8117f430ebc40c6c4ed784f2e', 'request_id': 'req-895b23be-3865-460a-80b9-adf3c3f7ccdf'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.706813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '748e57b8117f430ebc40c6c4ed784f2e', 'request_id': 'req-895b23be-3865-460a-80b9-adf3c3f7ccdf'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.727662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] The token cache is being invalidate because project 748e57b8117f430ebc40c6c4ed784f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:05.727973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.731674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-895b23be-3865-460a-80b9-adf3c3f7ccdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:05.735118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 917/1852] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:05 2022] DELETE /identity/v3/projects/748e57b8117f430ebc40c6c4ed784f2e => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:05.741398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d828-1ea2-49e0-98cd-b92b17fd7a55 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:05.741681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d828-1ea2-49e0-98cd-b92b17fd7a55 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:05.742663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d828-1ea2-49e0-98cd-b92b17fd7a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:05.767297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:05.781670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857b4bc-f25f-45a3-8028-adbb9d2bbc02 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:05.816555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-6433d828-1ea2-49e0-98cd-b92b17fd7a55 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:05.818313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6433d828-1ea2-49e0-98cd-b92b17fd7a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:05.898754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:05.899755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:05.900049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:05.900331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:05.901374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:05.903444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:05.919336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 936/1853] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:05.923945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:05.927184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:05.978902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 918/1854] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:05.987168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656260-cee8-408f-8aa5-c7a23467c077 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:05.987438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656260-cee8-408f-8aa5-c7a23467c077 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:05.987764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7656260-cee8-408f-8aa5-c7a23467c077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.029483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b7656260-cee8-408f-8aa5-c7a23467c077 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:06.030760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b7656260-cee8-408f-8aa5-c7a23467c077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:06.032910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfee5e-22d5-4949-b020-7fc07863d15c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:06.054752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 919/1855] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:06.058844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7656260-cee8-408f-8aa5-c7a23467c077 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:06.060287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05e3be-2ea6-495d-b26f-762f0ffe9d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:06.061336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05e3be-2ea6-495d-b26f-762f0ffe9d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.061624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05e3be-2ea6-495d-b26f-762f0ffe9d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.061967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05e3be-2ea6-495d-b26f-762f0ffe9d92 admin admin] PATH_INFO: `/v3/projects/1be5c03fb3ec45558b9c70c24618247d/users/29ef1ef69cbe481c949431154655eecd/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.077539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05e3be-2ea6-495d-b26f-762f0ffe9d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be5c03fb3ec45558b9c70c24618247d, user_id=29ef1ef69cbe481c949431154655eecd, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:06.079046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7656260-cee8-408f-8aa5-c7a23467c077 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:06.090728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05e3be-2ea6-495d-b26f-762f0ffe9d92 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:06.092464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:06.093211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.093493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.093804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.094567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:06.099144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:06.106790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:06.128875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 937/1856] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:05 2022] PUT /identity/v3/projects/1be5c03fb3ec45558b9c70c24618247d/users/29ef1ef69cbe481c949431154655eecd/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:06.140171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6e3e-8af4-49ca-8f44-2e7f0b5aec83 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.140543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6e3e-8af4-49ca-8f44-2e7f0b5aec83 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.140833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6e3e-8af4-49ca-8f44-2e7f0b5aec83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.141219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 920/1857] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:54:06.145487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:06.168167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55105f6-b0dd-4b44-9332-4d7523e74f49 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:06.242302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-99ba6e3e-8af4-49ca-8f44-2e7f0b5aec83 None None] MFA Rules not processed for user `29ef1ef69cbe481c949431154655eecd`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:06.247917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-99ba6e3e-8af4-49ca-8f44-2e7f0b5aec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:06.314675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16663ab-ec7a-4cc5-8576-a92134f5c518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:06.317095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16663ab-ec7a-4cc5-8576-a92134f5c518 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.318340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16663ab-ec7a-4cc5-8576-a92134f5c518 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.319648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16663ab-ec7a-4cc5-8576-a92134f5c518 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.323626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16663ab-ec7a-4cc5-8576-a92134f5c518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:06.327285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16663ab-ec7a-4cc5-8576-a92134f5c518 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:06.460163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 921/1858] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:06 2022] POST /identity/v3/projects => generated 377 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:54:06.463736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16663ab-ec7a-4cc5-8576-a92134f5c518 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:06.488502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 938/1859] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:06.503533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba6e3e-8af4-49ca-8f44-2e7f0b5aec83 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:06.505453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-99ba6e3e-8af4-49ca-8f44-2e7f0b5aec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:06.646814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:06.650323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16663ab-ec7a-4cc5-8576-a92134f5c518 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:06.668359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc76326-695a-48a8-a3f7-7a51a6544729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:06.669205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76326-695a-48a8-a3f7-7a51a6544729 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.669562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76326-695a-48a8-a3f7-7a51a6544729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.670495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc76326-695a-48a8-a3f7-7a51a6544729 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.670756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc76326-695a-48a8-a3f7-7a51a6544729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:06.673117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc76326-695a-48a8-a3f7-7a51a6544729 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:06.683511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba6e3e-8af4-49ca-8f44-2e7f0b5aec83 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:06.708470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:06.717401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 922/1860] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:06 2022] POST /identity/v3/users => generated 289 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:06.722645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc76326-695a-48a8-a3f7-7a51a6544729 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:06.741932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc76326-695a-48a8-a3f7-7a51a6544729 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:06.759616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbaa005-df16-4a1a-ad90-d43db9b0122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:06.760498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbaa005-df16-4a1a-ad90-d43db9b0122f admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.760914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbaa005-df16-4a1a-ad90-d43db9b0122f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.761380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbaa005-df16-4a1a-ad90-d43db9b0122f admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.763512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbaa005-df16-4a1a-ad90-d43db9b0122f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:06.764732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbaa005-df16-4a1a-ad90-d43db9b0122f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:06.800860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 923/1861] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:06.806912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbaa005-df16-4a1a-ad90-d43db9b0122f admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:06.832192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbaa005-df16-4a1a-ad90-d43db9b0122f admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:06.851105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a140d-362b-43f7-9576-aa45be01c98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:06.851920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a140d-362b-43f7-9576-aa45be01c98b admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.852440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a140d-362b-43f7-9576-aa45be01c98b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.852803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a140d-362b-43f7-9576-aa45be01c98b admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.853817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a140d-362b-43f7-9576-aa45be01c98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:06.856799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a140d-362b-43f7-9576-aa45be01c98b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:06.867710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:06.868612 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.869085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.869551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:06.893337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 924/1862] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:06.894411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a140d-362b-43f7-9576-aa45be01c98b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:06.917863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a140d-362b-43f7-9576-aa45be01c98b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:06.934974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8c16d-8123-46ba-bc09-6b56912da85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:06.935996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8c16d-8123-46ba-bc09-6b56912da85e admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:06.936356 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8c16d-8123-46ba-bc09-6b56912da85e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:06.936683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8c16d-8123-46ba-bc09-6b56912da85e admin admin] PATH_INFO: `/v3/projects/6621af629a454b97b6879705994e43ea/users/04c4bc187e474b5da94ef65b48c0778a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:07.005503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8c16d-8123-46ba-bc09-6b56912da85e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6621af629a454b97b6879705994e43ea, user_id=04c4bc187e474b5da94ef65b48c0778a, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:07.009240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8c16d-8123-46ba-bc09-6b56912da85e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:07.032968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 925/1863] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:06 2022] PUT /identity/v3/projects/6621af629a454b97b6879705994e43ea/users/04c4bc187e474b5da94ef65b48c0778a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:07.035409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:07.038536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:07.051984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32450144-42f1-4571-bf38-327d54bf97bd None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:07.052442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32450144-42f1-4571-bf38-327d54bf97bd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:07.052855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32450144-42f1-4571-bf38-327d54bf97bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:07.076392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 939/1864] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:07.110397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-32450144-42f1-4571-bf38-327d54bf97bd None None] MFA Rules not processed for user `04c4bc187e474b5da94ef65b48c0778a`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:07.110397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-32450144-42f1-4571-bf38-327d54bf97bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:07.111664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:07.113489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:07.169943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78718cc7-7afc-41a6-87c4-142e43b24e35 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:07.287617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 926/1865] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:07.294337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32450144-42f1-4571-bf38-327d54bf97bd None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:07.419473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:07.424133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:07.426206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:07.427747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] PATH_INFO: `/v3/projects/fe3c33a627154a87b3317cdd1d4c11f7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:07.432824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3c33a627154a87b3317cdd1d4c11f7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:07.440018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:07.609800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32450144-42f1-4571-bf38-327d54bf97bd None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:07.628364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3c33a627154a87b3317cdd1d4c11f7', 'request_id': 'req-cf07eafa-c8a1-4a78-b705-c3da709409d3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:07.648727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:07.659233 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3c33a627154a87b3317cdd1d4c11f7', 'request_id': 'req-cf07eafa-c8a1-4a78-b705-c3da709409d3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:07.676765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3c33a627154a87b3317cdd1d4c11f7', 'request_id': 'req-cf07eafa-c8a1-4a78-b705-c3da709409d3'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:07.715503 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] The token cache is being invalidate because project fe3c33a627154a87b3317cdd1d4c11f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:07.715850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:07.719941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-cf07eafa-c8a1-4a78-b705-c3da709409d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:07.728956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 940/1866] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:07 2022] DELETE /identity/v3/projects/fe3c33a627154a87b3317cdd1d4c11f7 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:07.730950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac096-5aaa-4793-8ed9-5a70ef22f2b8 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:07.732098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac096-5aaa-4793-8ed9-5a70ef22f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:07.732413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac096-5aaa-4793-8ed9-5a70ef22f2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:07.783448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:07.796610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-474ac096-5aaa-4793-8ed9-5a70ef22f2b8 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:07.798082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-474ac096-5aaa-4793-8ed9-5a70ef22f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:07.811127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:07.812252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:07.812645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:07.813092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:07.814031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:07.816137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:07.828373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 941/1867] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:07.834495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e0450-6728-44e9-aa33-0a3a18ebdea1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:07.834825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e0450-6728-44e9-aa33-0a3a18ebdea1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:07.835315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e0450-6728-44e9-aa33-0a3a18ebdea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:07.898965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f00e0450-6728-44e9-aa33-0a3a18ebdea1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:07.900504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f00e0450-6728-44e9-aa33-0a3a18ebdea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:07.921794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 942/1868] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:07.928470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e0450-6728-44e9-aa33-0a3a18ebdea1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:07.969636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 927/1869] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:07 2022] POST /identity/v3/projects => generated 375 bytes in 678 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:54:07.973341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e0450-6728-44e9-aa33-0a3a18ebdea1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:07.974643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:07.984320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:08.213269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2373d0-6df9-4607-9023-b1656cad73e2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.217745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.219262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.220736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.222261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.224190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:08.224750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.235219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.241510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb47820-469a-4df0-8255-46688bbb2785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.246436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb47820-469a-4df0-8255-46688bbb2785 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.246921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb47820-469a-4df0-8255-46688bbb2785 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.249892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb47820-469a-4df0-8255-46688bbb2785 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.252748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb47820-469a-4df0-8255-46688bbb2785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.255179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb47820-469a-4df0-8255-46688bbb2785 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.269677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 943/1870] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:54:08.274075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:08.296805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7a37e-0113-4e69-85b7-0aa6e4378893 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.310663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c727e52-17b8-4b35-9ea0-ec4cf32e7b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.314548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 928/1871] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:07 2022] POST /identity/v3/users => generated 294 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:08.315066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727e52-17b8-4b35-9ea0-ec4cf32e7b19 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.315534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727e52-17b8-4b35-9ea0-ec4cf32e7b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.315918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb47820-469a-4df0-8255-46688bbb2785 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:08.316239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727e52-17b8-4b35-9ea0-ec4cf32e7b19 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.323514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c727e52-17b8-4b35-9ea0-ec4cf32e7b19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.326238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c727e52-17b8-4b35-9ea0-ec4cf32e7b19 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.346702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb47820-469a-4df0-8255-46688bbb2785 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.369274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.370205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.370575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.370955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.374134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.380055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.412234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 929/1872] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:08.413965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:08.422669 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:08.422872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 944/1873] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:08 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:08.423869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c727e52-17b8-4b35-9ea0-ec4cf32e7b19 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:08.562513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c727e52-17b8-4b35-9ea0-ec4cf32e7b19 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.565084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fa558-35eb-46b5-a17c-96a838a72c2c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.569756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59475563-7808-4b1c-bcf4-6f94b3090009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.571404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475563-7808-4b1c-bcf4-6f94b3090009 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.571841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475563-7808-4b1c-bcf4-6f94b3090009 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.572440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59475563-7808-4b1c-bcf4-6f94b3090009 admin admin] PATH_INFO: `/v3/projects/b964173ff6b34e7eb9153970dd2b5b6c/users/531a0e8ad83b4b5e83b5e5b12b230d04/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.603872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59475563-7808-4b1c-bcf4-6f94b3090009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b964173ff6b34e7eb9153970dd2b5b6c, user_id=531a0e8ad83b4b5e83b5e5b12b230d04, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.607559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59475563-7808-4b1c-bcf4-6f94b3090009 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.645171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 945/1874] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:08 2022] PUT /identity/v3/projects/b964173ff6b34e7eb9153970dd2b5b6c/users/531a0e8ad83b4b5e83b5e5b12b230d04/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:08.645171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59475563-7808-4b1c-bcf4-6f94b3090009 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:08.701928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:08.703260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.705249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.706813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.707189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.715858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:08.766184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:08.801999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59475563-7808-4b1c-bcf4-6f94b3090009 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.819586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.820292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.820616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.821013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.821564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.823550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.852153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.856537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.874990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 946/1875] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:08 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:08.876911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:08.879377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:08.880696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 930/1876] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:08.881588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:08.900526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59727f6a-829e-475c-99a8-eb5f2e2e658a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.908319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.909946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.910263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.911206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] PATH_INFO: `/v3/projects/92782533b4c44f1bbc7b425a768ab6e6` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.911394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95bfa8-ba93-4072-9e4f-9eec42c2912f service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.913956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92782533b4c44f1bbc7b425a768ab6e6)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.916795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.922077 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8fe72-0eac-4211-94c0-fc11303d06f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.922746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8fe72-0eac-4211-94c0-fc11303d06f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.923030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8fe72-0eac-4211-94c0-fc11303d06f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.923309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8fe72-0eac-4211-94c0-fc11303d06f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.924043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8fe72-0eac-4211-94c0-fc11303d06f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.934608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8fe72-0eac-4211-94c0-fc11303d06f2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:08.949387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 931/1877] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:08.959666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8fe72-0eac-4211-94c0-fc11303d06f2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:08.975920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8fe72-0eac-4211-94c0-fc11303d06f2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:08.991896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392595d-200e-45b5-ae46-126fc484c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:08.992912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392595d-200e-45b5-ae46-126fc484c057 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:08.993306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392595d-200e-45b5-ae46-126fc484c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:08.993582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392595d-200e-45b5-ae46-126fc484c057 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:08.994516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392595d-200e-45b5-ae46-126fc484c057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:08.996904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392595d-200e-45b5-ae46-126fc484c057 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:09.032278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 932/1878] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:09.032278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392595d-200e-45b5-ae46-126fc484c057 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:09.050220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392595d-200e-45b5-ae46-126fc484c057 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:09.083826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92782533b4c44f1bbc7b425a768ab6e6', 'request_id': 'req-89930ead-17e0-44ed-9622-4f34d9c0af20'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:09.112819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92782533b4c44f1bbc7b425a768ab6e6', 'request_id': 'req-89930ead-17e0-44ed-9622-4f34d9c0af20'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:09.125889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92782533b4c44f1bbc7b425a768ab6e6', 'request_id': 'req-89930ead-17e0-44ed-9622-4f34d9c0af20'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:09.166615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] The token cache is being invalidate because project 92782533b4c44f1bbc7b425a768ab6e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:09.167648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:09.170527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:09.173930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 947/1879] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:08 2022] DELETE /identity/v3/projects/92782533b4c44f1bbc7b425a768ab6e6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:09.182198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:09.187726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:09.204795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad507f-1e28-45e8-97b2-d5e3164ac3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:09.205794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad507f-1e28-45e8-97b2-d5e3164ac3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:09.206087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad507f-1e28-45e8-97b2-d5e3164ac3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:09.206367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad507f-1e28-45e8-97b2-d5e3164ac3bc admin admin] PATH_INFO: `/v3/projects/b964173ff6b34e7eb9153970dd2b5b6c/users/531a0e8ad83b4b5e83b5e5b12b230d04/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:09.215182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad507f-1e28-45e8-97b2-d5e3164ac3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b964173ff6b34e7eb9153970dd2b5b6c, user_id=531a0e8ad83b4b5e83b5e5b12b230d04, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:09.218064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad507f-1e28-45e8-97b2-d5e3164ac3bc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:09.229802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89930ead-17e0-44ed-9622-4f34d9c0af20 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:09.244863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78482d3-a88b-40e0-9a95-c94ca5a1fd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:09.245289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78482d3-a88b-40e0-9a95-c94ca5a1fd77 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:09.245588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78482d3-a88b-40e0-9a95-c94ca5a1fd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:09.245910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78482d3-a88b-40e0-9a95-c94ca5a1fd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:09.248515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78482d3-a88b-40e0-9a95-c94ca5a1fd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:09.251754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78482d3-a88b-40e0-9a95-c94ca5a1fd77 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:09.257631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 933/1880] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:09 2022] PUT /identity/v3/projects/b964173ff6b34e7eb9153970dd2b5b6c/users/531a0e8ad83b4b5e83b5e5b12b230d04/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:09.280640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c06e2-ed17-4e98-a4c3-044703badeff None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:09.280640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c06e2-ed17-4e98-a4c3-044703badeff None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:09.280640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c06e2-ed17-4e98-a4c3-044703badeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:09.338849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 948/1881] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:09.340705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91a6d-78bc-4261-a23d-f1c0bc9c7457 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:09.344541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91a6d-78bc-4261-a23d-f1c0bc9c7457 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:09.346588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef91a6d-78bc-4261-a23d-f1c0bc9c7457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:09.378911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-498c06e2-ed17-4e98-a4c3-044703badeff None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:09.391446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-498c06e2-ed17-4e98-a4c3-044703badeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:09.468320 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-bef91a6d-78bc-4261-a23d-f1c0bc9c7457 None None] MFA Rules not processed for user `531a0e8ad83b4b5e83b5e5b12b230d04`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:09.469448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bef91a6d-78bc-4261-a23d-f1c0bc9c7457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:09.691777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 934/1882] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:09.692964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c06e2-ed17-4e98-a4c3-044703badeff None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:09.724786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c06e2-ed17-4e98-a4c3-044703badeff None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:09.753286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:09.757767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:09.758282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:09.758599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] PATH_INFO: `/v3/projects/bf84152541c84ce38d97c48042cbc581/users/86bb9222468d4042a60c03a840844859/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:09.792317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 949/1883] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:09.793915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddacc23-3f97-4ea4-bb07-35edede8d88b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:09.796175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddacc23-3f97-4ea4-bb07-35edede8d88b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:09.796453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddacc23-3f97-4ea4-bb07-35edede8d88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:09.819704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf84152541c84ce38d97c48042cbc581, user_id=86bb9222468d4042a60c03a840844859, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:09.822976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:09.845032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1ddacc23-3f97-4ea4-bb07-35edede8d88b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:09.847419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1ddacc23-3f97-4ea4-bb07-35edede8d88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:09.855632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 935/1884] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:09 2022] PUT /identity/v3/projects/bf84152541c84ce38d97c48042cbc581/users/86bb9222468d4042a60c03a840844859/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:09.857269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:09.860184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:09.868351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:09.900491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:09.986071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 950/1885] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:09.988217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2532-77fa-4fbc-a115-a729a5bb587a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:09.988761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2532-77fa-4fbc-a115-a729a5bb587a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:09.989719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2532-77fa-4fbc-a115-a729a5bb587a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.056774 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f5ba2532-77fa-4fbc-a115-a729a5bb587a None None] MFA Rules not processed for user `86bb9222468d4042a60c03a840844859`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:10.060527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f5ba2532-77fa-4fbc-a115-a729a5bb587a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:10.063094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd4ebf-9a53-464d-9a4f-f6aae84be6de admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:10.083202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce048a46-2416-4664-936d-41c784a77429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:10.083202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce048a46-2416-4664-936d-41c784a77429 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.083202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce048a46-2416-4664-936d-41c784a77429 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.083202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce048a46-2416-4664-936d-41c784a77429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.083564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ce048a46-2416-4664-936d-41c784a77429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:10.274347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 951/1886] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:10.276574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2a5a-e596-4c14-884a-01d80cdedded None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.277131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2a5a-e596-4c14-884a-01d80cdedded None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.278118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2a5a-e596-4c14-884a-01d80cdedded None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.285449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 952/1887] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:54:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:54:10.286678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b2a5a-e596-4c14-884a-01d80cdedded None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:10.287198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce048a46-2416-4664-936d-41c784a77429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:10.289245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce048a46-2416-4664-936d-41c784a77429 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:10.334741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b2a5a-e596-4c14-884a-01d80cdedded None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:10.345990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 936/1888] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:10.346892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8520-6bc6-4460-8e0e-677e31e9710e None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.347976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8520-6bc6-4460-8e0e-677e31e9710e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.348603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8520-6bc6-4460-8e0e-677e31e9710e None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.351527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 937/1889] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:54:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:54:10.353052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8520-6bc6-4460-8e0e-677e31e9710e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:10.374534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa96c75-d446-4003-956f-b3bd411ee4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:10.375640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa96c75-d446-4003-956f-b3bd411ee4de admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.376147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa96c75-d446-4003-956f-b3bd411ee4de admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.376260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8520-6bc6-4460-8e0e-677e31e9710e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:10.376817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa96c75-d446-4003-956f-b3bd411ee4de admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.381349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa96c75-d446-4003-956f-b3bd411ee4de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:10.407650 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd3a8-de83-48bf-9645-8fa23d65e476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:10.408332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd3a8-de83-48bf-9645-8fa23d65e476 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.408612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd3a8-de83-48bf-9645-8fa23d65e476 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.408885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dd3a8-de83-48bf-9645-8fa23d65e476 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.419324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa96c75-d446-4003-956f-b3bd411ee4de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:10.427413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa96c75-d446-4003-956f-b3bd411ee4de admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:10.431457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dd3a8-de83-48bf-9645-8fa23d65e476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:10.435096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dd3a8-de83-48bf-9645-8fa23d65e476 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:10.489633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 953/1890] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:54:10.492628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3839e-3dab-402a-96bd-7d4e2f4fa443 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.492920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3839e-3dab-402a-96bd-7d4e2f4fa443 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.494360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3839e-3dab-402a-96bd-7d4e2f4fa443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.499228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 938/1891] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:10 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:10.500499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd3a8-de83-48bf-9645-8fa23d65e476 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:10.558059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b8a3839e-3dab-402a-96bd-7d4e2f4fa443 None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:10.561931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b8a3839e-3dab-402a-96bd-7d4e2f4fa443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:10.657886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dd3a8-de83-48bf-9645-8fa23d65e476 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:10.695900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfef5b8-0646-4e3b-81d7-9a4b37725cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:10.697488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfef5b8-0646-4e3b-81d7-9a4b37725cbd admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.697867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfef5b8-0646-4e3b-81d7-9a4b37725cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.698129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfef5b8-0646-4e3b-81d7-9a4b37725cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.699140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfef5b8-0646-4e3b-81d7-9a4b37725cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:10.702200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfef5b8-0646-4e3b-81d7-9a4b37725cbd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:10.800890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 939/1892] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:10 2022] POST /identity/v3/projects => generated 369 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:10.802116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfef5b8-0646-4e3b-81d7-9a4b37725cbd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:10.817382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 954/1893] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:54:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:10.818773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4493-2742-42d6-a9e5-3b1c16cc6b6d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.819114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4493-2742-42d6-a9e5-3b1c16cc6b6d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.819393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4493-2742-42d6-a9e5-3b1c16cc6b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.888345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:10.914148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-427b4493-2742-42d6-a9e5-3b1c16cc6b6d None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:10.920356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-427b4493-2742-42d6-a9e5-3b1c16cc6b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:10.932487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfef5b8-0646-4e3b-81d7-9a4b37725cbd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:10.938556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff633-1936-42fd-baf0-090d1a507e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:10.939547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff633-1936-42fd-baf0-090d1a507e19 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:10.939945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff633-1936-42fd-baf0-090d1a507e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:10.940475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff633-1936-42fd-baf0-090d1a507e19 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:10.941294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ff633-1936-42fd-baf0-090d1a507e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:10.943806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ff633-1936-42fd-baf0-090d1a507e19 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:10.978808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 940/1894] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:10 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:10.983602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff633-1936-42fd-baf0-090d1a507e19 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:10.987973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:11.002078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff633-1936-42fd-baf0-090d1a507e19 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:11.008037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9704f-3f7d-4158-a9f2-173331dfc2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:11.009116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9704f-3f7d-4158-a9f2-173331dfc2fd admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.009535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9704f-3f7d-4158-a9f2-173331dfc2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.010032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9704f-3f7d-4158-a9f2-173331dfc2fd admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.016166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9704f-3f7d-4158-a9f2-173331dfc2fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:11.025410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9704f-3f7d-4158-a9f2-173331dfc2fd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:11.068324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 941/1895] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:10 2022] POST /identity/v3/users => generated 285 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:11.074870 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9704f-3f7d-4158-a9f2-173331dfc2fd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:11.093098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9704f-3f7d-4158-a9f2-173331dfc2fd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:11.133077 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889473c5-fa4f-4939-ae2f-27ee65f84755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:11.135613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889473c5-fa4f-4939-ae2f-27ee65f84755 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.136200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889473c5-fa4f-4939-ae2f-27ee65f84755 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.136882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889473c5-fa4f-4939-ae2f-27ee65f84755 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.138520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889473c5-fa4f-4939-ae2f-27ee65f84755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:11.141157 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889473c5-fa4f-4939-ae2f-27ee65f84755 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:11.182972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 955/1896] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:54:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:11.184002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b4493-2742-42d6-a9e5-3b1c16cc6b6d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:11.186216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 942/1897] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:11.191246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889473c5-fa4f-4939-ae2f-27ee65f84755 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:11.211625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b4493-2742-42d6-a9e5-3b1c16cc6b6d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:11.216699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889473c5-fa4f-4939-ae2f-27ee65f84755 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:11.232550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5621792-a51b-4f0f-bbce-3e8f659704a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:11.233662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621792-a51b-4f0f-bbce-3e8f659704a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.245765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2bdff-14c3-4864-bcb3-be7fb30d73a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:11.263373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621792-a51b-4f0f-bbce-3e8f659704a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.263373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621792-a51b-4f0f-bbce-3e8f659704a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.264061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2bdff-14c3-4864-bcb3-be7fb30d73a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.264150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5621792-a51b-4f0f-bbce-3e8f659704a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:11.264264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2bdff-14c3-4864-bcb3-be7fb30d73a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.264264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2bdff-14c3-4864-bcb3-be7fb30d73a7 admin admin] PATH_INFO: `/v3/projects/a3464367f0264ee29073545f8217bc50/users/e918b5151c07444d8240264ce7cbaf21/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.264403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5621792-a51b-4f0f-bbce-3e8f659704a3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:11.298916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2bdff-14c3-4864-bcb3-be7fb30d73a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3464367f0264ee29073545f8217bc50, user_id=e918b5151c07444d8240264ce7cbaf21, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:11.303055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 956/1898] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:11.308513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2bdff-14c3-4864-bcb3-be7fb30d73a7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:11.308715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5621792-a51b-4f0f-bbce-3e8f659704a3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:11.336655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 943/1899] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:11 2022] PUT /identity/v3/projects/a3464367f0264ee29073545f8217bc50/users/e918b5151c07444d8240264ce7cbaf21/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:11.340772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2bdff-14c3-4864-bcb3-be7fb30d73a7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:11.345638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5621792-a51b-4f0f-bbce-3e8f659704a3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:11.507717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:11.508496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.508852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.509205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.510096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:11.511924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:11.532370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 957/1900] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:11.532370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:11.534956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2bdff-14c3-4864-bcb3-be7fb30d73a7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:11.536314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:11.549363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1d36e-373c-480c-8b08-ce2ce13964cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:11.550553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d36e-373c-480c-8b08-ce2ce13964cb admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.551508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d36e-373c-480c-8b08-ce2ce13964cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.552612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1d36e-373c-480c-8b08-ce2ce13964cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.553843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1d36e-373c-480c-8b08-ce2ce13964cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:11.559521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1d36e-373c-480c-8b08-ce2ce13964cb admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:11.592771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 944/1901] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:11.597934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1d36e-373c-480c-8b08-ce2ce13964cb admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:11.630180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1d36e-373c-480c-8b08-ce2ce13964cb admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:11.650493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492bcde-be1e-44e3-8ed7-efdfefe0031a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:11.651977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492bcde-be1e-44e3-8ed7-efdfefe0031a admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.652429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492bcde-be1e-44e3-8ed7-efdfefe0031a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.652848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492bcde-be1e-44e3-8ed7-efdfefe0031a admin admin] PATH_INFO: `/v3/projects/7020a48c951347c6ae1d177b5e5722c6/users/7f4c57aa64f14612a634373bfe0b5f5f/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.675005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f431b5-1222-44fb-93af-0ec6c12c1057 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:11.689122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492bcde-be1e-44e3-8ed7-efdfefe0031a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7020a48c951347c6ae1d177b5e5722c6, user_id=7f4c57aa64f14612a634373bfe0b5f5f, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:11.690180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef05147-484b-40f0-8636-6f90f93bfcc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:11.691006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef05147-484b-40f0-8636-6f90f93bfcc1 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.691294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef05147-484b-40f0-8636-6f90f93bfcc1 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.691629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef05147-484b-40f0-8636-6f90f93bfcc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.694821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492bcde-be1e-44e3-8ed7-efdfefe0031a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:11.721560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef05147-484b-40f0-8636-6f90f93bfcc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:11.735983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef05147-484b-40f0-8636-6f90f93bfcc1 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:11.736778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 945/1902] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:11 2022] PUT /identity/v3/projects/7020a48c951347c6ae1d177b5e5722c6/users/7f4c57aa64f14612a634373bfe0b5f5f/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:11.738617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492bcde-be1e-44e3-8ed7-efdfefe0031a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:11.755339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 958/1903] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:11.756527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23939b-b46c-43be-810e-3c4dc1e24c8b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:11.756803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23939b-b46c-43be-810e-3c4dc1e24c8b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:11.757082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23939b-b46c-43be-810e-3c4dc1e24c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:11.866313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3d23939b-b46c-43be-810e-3c4dc1e24c8b None None] MFA Rules not processed for user `7f4c57aa64f14612a634373bfe0b5f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:11.871246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3d23939b-b46c-43be-810e-3c4dc1e24c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:11.903396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:11.926546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:11.995211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492bcde-be1e-44e3-8ed7-efdfefe0031a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:12.013218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:12.015457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:12.015748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:12.017982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] PATH_INFO: `/v3/projects/a3464367f0264ee29073545f8217bc50/users/e918b5151c07444d8240264ce7cbaf21/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:12.023468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3464367f0264ee29073545f8217bc50, user_id=e918b5151c07444d8240264ce7cbaf21, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:12.035333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:12.098181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 946/1904] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:11 2022] PUT /identity/v3/projects/a3464367f0264ee29073545f8217bc50/users/e918b5151c07444d8240264ce7cbaf21/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:12.102152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:12.121843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 959/1905] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:12.121843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be925-6545-4887-90d7-65c019a71a88 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:12.121843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be925-6545-4887-90d7-65c019a71a88 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:12.121843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659be925-6545-4887-90d7-65c019a71a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:12.124773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:12.231312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-659be925-6545-4887-90d7-65c019a71a88 None None] MFA Rules not processed for user `e918b5151c07444d8240264ce7cbaf21`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:12.235869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-659be925-6545-4887-90d7-65c019a71a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:12.412601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc3090-f1c5-4596-9d07-c25879a687cc admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:12.430569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06e7c9-dbf3-4376-9a63-97c55780bdc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:12.431247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e7c9-dbf3-4376-9a63-97c55780bdc5 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:12.431528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e7c9-dbf3-4376-9a63-97c55780bdc5 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:12.431806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06e7c9-dbf3-4376-9a63-97c55780bdc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:12.454295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06e7c9-dbf3-4376-9a63-97c55780bdc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:12.455933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06e7c9-dbf3-4376-9a63-97c55780bdc5 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:12.485852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 947/1906] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:12.486990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06e7c9-dbf3-4376-9a63-97c55780bdc5 service nova] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:12.490956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 960/1907] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:12.684832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06e7c9-dbf3-4376-9a63-97c55780bdc5 service nova] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:12.700511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b17e99-76d8-4e1b-90df-6ba8c885e1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:12.701403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b17e99-76d8-4e1b-90df-6ba8c885e1ec admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:12.702751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b17e99-76d8-4e1b-90df-6ba8c885e1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:12.703311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b17e99-76d8-4e1b-90df-6ba8c885e1ec admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:12.706157 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b17e99-76d8-4e1b-90df-6ba8c885e1ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:12.711409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659be925-6545-4887-90d7-65c019a71a88 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:12.721294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b17e99-76d8-4e1b-90df-6ba8c885e1ec admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:12.795371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 948/1908] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:12 2022] POST /identity/v3/projects => generated 365 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:12.801478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b17e99-76d8-4e1b-90df-6ba8c885e1ec admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:12.862768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659be925-6545-4887-90d7-65c019a71a88 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:12.889951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:12.917228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b17e99-76d8-4e1b-90df-6ba8c885e1ec admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:12.923448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81060e9-df07-42ae-954c-32f15833d84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:12.924523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81060e9-df07-42ae-954c-32f15833d84e admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:12.924903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81060e9-df07-42ae-954c-32f15833d84e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:12.925274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81060e9-df07-42ae-954c-32f15833d84e admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:12.926457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81060e9-df07-42ae-954c-32f15833d84e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:12.929580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81060e9-df07-42ae-954c-32f15833d84e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:12.985416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 949/1909] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:12 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:12.986845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f626566-672d-43ce-b82d-0b0116576dc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:12.987538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f626566-672d-43ce-b82d-0b0116576dc1 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:12.987903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f626566-672d-43ce-b82d-0b0116576dc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:12.988308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f626566-672d-43ce-b82d-0b0116576dc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:12.990677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81060e9-df07-42ae-954c-32f15833d84e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:13.002400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:13.018861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81060e9-df07-42ae-954c-32f15833d84e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:13.023945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedb96f-821f-4ee8-9754-ed65d53134b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:13.024635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedb96f-821f-4ee8-9754-ed65d53134b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.035317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedb96f-821f-4ee8-9754-ed65d53134b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.035826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedb96f-821f-4ee8-9754-ed65d53134b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.036666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eedb96f-821f-4ee8-9754-ed65d53134b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:13.038897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eedb96f-821f-4ee8-9754-ed65d53134b4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:13.050211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 950/1910] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:13.055867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedb96f-821f-4ee8-9754-ed65d53134b4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:13.068534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedb96f-821f-4ee8-9754-ed65d53134b4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:13.073766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cff50a-2c50-4c6c-b2ed-dbbe12fd265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:13.074642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cff50a-2c50-4c6c-b2ed-dbbe12fd265c admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.074929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cff50a-2c50-4c6c-b2ed-dbbe12fd265c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.075256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cff50a-2c50-4c6c-b2ed-dbbe12fd265c admin admin] PATH_INFO: `/v3/projects/e328df768c58498581c53a9d3f766a6a/users/e563a5bcfb9d40939ffbf53596f45f66/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.083044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f626566-672d-43ce-b82d-0b0116576dc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:13.084825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f626566-672d-43ce-b82d-0b0116576dc1 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:13.096515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cff50a-2c50-4c6c-b2ed-dbbe12fd265c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e328df768c58498581c53a9d3f766a6a, user_id=e563a5bcfb9d40939ffbf53596f45f66, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:13.097819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 961/1911] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:13.099743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cff50a-2c50-4c6c-b2ed-dbbe12fd265c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:13.117505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 951/1912] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:13 2022] PUT /identity/v3/projects/e328df768c58498581c53a9d3f766a6a/users/e563a5bcfb9d40939ffbf53596f45f66/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:13.121295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f626566-672d-43ce-b82d-0b0116576dc1 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:13.132792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f626566-672d-43ce-b82d-0b0116576dc1 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:13.197576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8e5d8-8c7f-4882-b6eb-a8716d1940a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:13.199183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e5d8-8c7f-4882-b6eb-a8716d1940a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.199183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e5d8-8c7f-4882-b6eb-a8716d1940a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.199374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cff50a-2c50-4c6c-b2ed-dbbe12fd265c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:13.199479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8e5d8-8c7f-4882-b6eb-a8716d1940a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.199777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8e5d8-8c7f-4882-b6eb-a8716d1940a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:13.201443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8e5d8-8c7f-4882-b6eb-a8716d1940a4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:13.213956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 962/1913] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:13.218006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8e5d8-8c7f-4882-b6eb-a8716d1940a4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:13.229255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8e5d8-8c7f-4882-b6eb-a8716d1940a4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:13.239620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9a8de-415e-4933-b3bb-30f8565e1333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:13.240558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a8de-415e-4933-b3bb-30f8565e1333 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.240837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a8de-415e-4933-b3bb-30f8565e1333 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.241110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9a8de-415e-4933-b3bb-30f8565e1333 admin admin] PATH_INFO: `/v3/projects/e328df768c58498581c53a9d3f766a6a/users/e563a5bcfb9d40939ffbf53596f45f66/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.252025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9a8de-415e-4933-b3bb-30f8565e1333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e328df768c58498581c53a9d3f766a6a, user_id=e563a5bcfb9d40939ffbf53596f45f66, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:13.255697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9a8de-415e-4933-b3bb-30f8565e1333 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:13.269814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 963/1914] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:13 2022] PUT /identity/v3/projects/e328df768c58498581c53a9d3f766a6a/users/e563a5bcfb9d40939ffbf53596f45f66/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:13.276013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cff50a-2c50-4c6c-b2ed-dbbe12fd265c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:13.280443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e7af-4f78-43e9-a778-09d10e4d50b5 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.280746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e7af-4f78-43e9-a778-09d10e4d50b5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.281403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e7af-4f78-43e9-a778-09d10e4d50b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.317684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-84c4e7af-4f78-43e9-a778-09d10e4d50b5 None None] MFA Rules not processed for user `e563a5bcfb9d40939ffbf53596f45f66`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:13.319160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-84c4e7af-4f78-43e9-a778-09d10e4d50b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:13.351656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a49bc2-53c9-424b-8e55-8d5004166dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:13.352332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a49bc2-53c9-424b-8e55-8d5004166dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.352620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a49bc2-53c9-424b-8e55-8d5004166dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.352920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a49bc2-53c9-424b-8e55-8d5004166dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.401364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 964/1915] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:13.412901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4e7af-4f78-43e9-a778-09d10e4d50b5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:13.414397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-84c4e7af-4f78-43e9-a778-09d10e4d50b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:13.418362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a49bc2-53c9-424b-8e55-8d5004166dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:13.419820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a49bc2-53c9-424b-8e55-8d5004166dd1 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:13.427758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4e7af-4f78-43e9-a778-09d10e4d50b5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:13.430899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 952/1916] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:13.431667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80ce7e-9135-4e71-a9b8-30673d3e2886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:13.432465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80ce7e-9135-4e71-a9b8-30673d3e2886 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.432805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80ce7e-9135-4e71-a9b8-30673d3e2886 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.433590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80ce7e-9135-4e71-a9b8-30673d3e2886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.441488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:13.444084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80ce7e-9135-4e71-a9b8-30673d3e2886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:13.445551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80ce7e-9135-4e71-a9b8-30673d3e2886 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:13.458419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 965/1917] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:13.716860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a49bc2-53c9-424b-8e55-8d5004166dd1 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:13.727170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:13.789030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a49bc2-53c9-424b-8e55-8d5004166dd1 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:13.794839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:13.795804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.796153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.796493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] PATH_INFO: `/v3/users/86bb9222468d4042a60c03a840844859` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.797756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86bb9222468d4042a60c03a840844859)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:13.799783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:13.850369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86bb9222468d4042a60c03a840844859'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:13.865633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86bb9222468d4042a60c03a840844859'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:13.874408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86bb9222468d4042a60c03a840844859'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:13.883137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86bb9222468d4042a60c03a840844859'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:13.883941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86bb9222468d4042a60c03a840844859'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:13.890317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 953/1918] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:13 2022] DELETE /identity/v3/users/86bb9222468d4042a60c03a840844859 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:13.895716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159ebc2-b8f2-4fcb-b8bb-4706c324ee39 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:13.896158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159ebc2-b8f2-4fcb-b8bb-4706c324ee39 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:13.896268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159ebc2-b8f2-4fcb-b8bb-4706c324ee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:13.915833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3159ebc2-b8f2-4fcb-b8bb-4706c324ee39 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:13.917301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3159ebc2-b8f2-4fcb-b8bb-4706c324ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:13.994615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 966/1919] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:14.006044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:14.007463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:14.085453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6b37-5d34-4924-b5b8-242d61f48c67 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:14.093454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92589b16-6445-4e4d-92ec-ba63f27c9b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:14.093454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92589b16-6445-4e4d-92ec-ba63f27c9b30 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:14.093454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92589b16-6445-4e4d-92ec-ba63f27c9b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:14.093454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92589b16-6445-4e4d-92ec-ba63f27c9b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:14.101442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92589b16-6445-4e4d-92ec-ba63f27c9b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:14.102883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92589b16-6445-4e4d-92ec-ba63f27c9b30 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:14.113659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 954/1920] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:14.438970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159ebc2-b8f2-4fcb-b8bb-4706c324ee39 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:14.444103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3159ebc2-b8f2-4fcb-b8bb-4706c324ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:14.458773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159ebc2-b8f2-4fcb-b8bb-4706c324ee39 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:14.466584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:14.467470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:14.467758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:14.468031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] PATH_INFO: `/v3/projects/bf84152541c84ce38d97c48042cbc581` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:14.469283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf84152541c84ce38d97c48042cbc581)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:14.472005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:14.546447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf84152541c84ce38d97c48042cbc581', 'request_id': 'req-d383640c-2611-469d-971b-f74cf34df981'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.561177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf84152541c84ce38d97c48042cbc581', 'request_id': 'req-d383640c-2611-469d-971b-f74cf34df981'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.574367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf84152541c84ce38d97c48042cbc581', 'request_id': 'req-d383640c-2611-469d-971b-f74cf34df981'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.588410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] The token cache is being invalidate because project bf84152541c84ce38d97c48042cbc581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:14.588915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.589800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.597159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 967/1921] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:14 2022] DELETE /identity/v3/projects/bf84152541c84ce38d97c48042cbc581 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:14.602634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92589b16-6445-4e4d-92ec-ba63f27c9b30 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:14.616552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92589b16-6445-4e4d-92ec-ba63f27c9b30 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:14.656559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:14.702783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:14.703820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:14.704164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:14.704503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] PATH_INFO: `/v3/users/e918b5151c07444d8240264ce7cbaf21` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:14.705906 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e918b5151c07444d8240264ce7cbaf21)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:14.708907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:14.766619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e918b5151c07444d8240264ce7cbaf21'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.779506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e918b5151c07444d8240264ce7cbaf21'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.788235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e918b5151c07444d8240264ce7cbaf21'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.796408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e918b5151c07444d8240264ce7cbaf21'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.797337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e918b5151c07444d8240264ce7cbaf21'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:14.804809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 955/1922] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:14 2022] DELETE /identity/v3/users/e918b5151c07444d8240264ce7cbaf21 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:15.042940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.044597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.057665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383640c-2611-469d-971b-f74cf34df981 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.128107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.129284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.129847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.130048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] PATH_INFO: `/v3/users/7f4c57aa64f14612a634373bfe0b5f5f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.131338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4c57aa64f14612a634373bfe0b5f5f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.133514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.141394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.142889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.161053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad86f52-7a1a-4603-9658-fa5c5653a350 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.161689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:15.167958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.169140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.169581 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.170058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] PATH_INFO: `/v3/projects/a3464367f0264ee29073545f8217bc50` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.171343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3464367f0264ee29073545f8217bc50)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.173936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.196348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4c57aa64f14612a634373bfe0b5f5f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.210375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4c57aa64f14612a634373bfe0b5f5f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.218049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4c57aa64f14612a634373bfe0b5f5f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.218675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4c57aa64f14612a634373bfe0b5f5f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.225890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b48a4923-3060-4a68-806f-81c4834bceda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4c57aa64f14612a634373bfe0b5f5f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.228239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3464367f0264ee29073545f8217bc50', 'request_id': 'req-24f15856-d2ef-426e-823d-748767a39ea6'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.231900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 968/1923] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:15 2022] DELETE /identity/v3/users/7f4c57aa64f14612a634373bfe0b5f5f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:15.234986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3464367f0264ee29073545f8217bc50', 'request_id': 'req-24f15856-d2ef-426e-823d-748767a39ea6'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.236885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51e387-09cb-4763-bc30-f3b6aa429ca1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.237179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51e387-09cb-4763-bc30-f3b6aa429ca1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.237463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51e387-09cb-4763-bc30-f3b6aa429ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.245872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3464367f0264ee29073545f8217bc50', 'request_id': 'req-24f15856-d2ef-426e-823d-748767a39ea6'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.258015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] The token cache is being invalidate because project a3464367f0264ee29073545f8217bc50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:15.258297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.258487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1d51e387-09cb-4763-bc30-f3b6aa429ca1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:15.258961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-24f15856-d2ef-426e-823d-748767a39ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:15.259740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1d51e387-09cb-4763-bc30-f3b6aa429ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.260525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 956/1924] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:15 2022] DELETE /identity/v3/projects/a3464367f0264ee29073545f8217bc50 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:15.265210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e339c4-534a-4780-a43d-da4d1a389481 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.265590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e339c4-534a-4780-a43d-da4d1a389481 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.266004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e339c4-534a-4780-a43d-da4d1a389481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.285170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b4e339c4-534a-4780-a43d-da4d1a389481 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:15.286439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b4e339c4-534a-4780-a43d-da4d1a389481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.317984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:15.324102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 969/1925] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:15.333946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51e387-09cb-4763-bc30-f3b6aa429ca1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.335469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1d51e387-09cb-4763-bc30-f3b6aa429ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.352396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 957/1926] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:15.360834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd2762-08f6-450c-ba93-f1f1c6e86867 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.361027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd2762-08f6-450c-ba93-f1f1c6e86867 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.361342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd2762-08f6-450c-ba93-f1f1c6e86867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.379179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-6bcd2762-08f6-450c-ba93-f1f1c6e86867 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:15.380390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6bcd2762-08f6-450c-ba93-f1f1c6e86867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.388812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 958/1927] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:15.393081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd2762-08f6-450c-ba93-f1f1c6e86867 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.395104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51e387-09cb-4763-bc30-f3b6aa429ca1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.398876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebed77-0681-4d1c-8ea4-6861ff0236e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.399540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebed77-0681-4d1c-8ea4-6861ff0236e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.399820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebed77-0681-4d1c-8ea4-6861ff0236e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.400094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebed77-0681-4d1c-8ea4-6861ff0236e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.402621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd2762-08f6-450c-ba93-f1f1c6e86867 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.407309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.407996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.408273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.408547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.409271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:15.409588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebed77-0681-4d1c-8ea4-6861ff0236e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.409693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.411115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebed77-0681-4d1c-8ea4-6861ff0236e0 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.411592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.420863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 970/1928] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:15.424821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 959/1929] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:54:15.429468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebed77-0681-4d1c-8ea4-6861ff0236e0 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.439216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebed77-0681-4d1c-8ea4-6861ff0236e0 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.444079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc956a1a-f3a8-4c49-8fef-b0067b272fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.444742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc956a1a-f3a8-4c49-8fef-b0067b272fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.445147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc956a1a-f3a8-4c49-8fef-b0067b272fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.445433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc956a1a-f3a8-4c49-8fef-b0067b272fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.446190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc956a1a-f3a8-4c49-8fef-b0067b272fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.448124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc956a1a-f3a8-4c49-8fef-b0067b272fc7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.478923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 971/1930] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:15 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:15.480371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.492086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c17876-6641-444b-b70b-adf57beba99a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.553467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c553a5d-25a6-4c72-b932-4d866d647cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.554202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553a5d-25a6-4c72-b932-4d866d647cad admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.554641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553a5d-25a6-4c72-b932-4d866d647cad admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.554922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c553a5d-25a6-4c72-b932-4d866d647cad admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.555942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c553a5d-25a6-4c72-b932-4d866d647cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.558067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c553a5d-25a6-4c72-b932-4d866d647cad admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.593023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 960/1931] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:15 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:15.598601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc956a1a-f3a8-4c49-8fef-b0067b272fc7 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.616873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc956a1a-f3a8-4c49-8fef-b0067b272fc7 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.623818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ad65f-5a32-46ae-a8ef-151960e66b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.624596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ad65f-5a32-46ae-a8ef-151960e66b30 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.624945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ad65f-5a32-46ae-a8ef-151960e66b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.625282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ad65f-5a32-46ae-a8ef-151960e66b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.626127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ad65f-5a32-46ae-a8ef-151960e66b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.627814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ad65f-5a32-46ae-a8ef-151960e66b30 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.641896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 972/1932] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:15.646380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c553a5d-25a6-4c72-b932-4d866d647cad admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.657501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c553a5d-25a6-4c72-b932-4d866d647cad admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.665056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9af72-cd52-4ea4-82c9-1fddd6c03397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.666023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9af72-cd52-4ea4-82c9-1fddd6c03397 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.666518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9af72-cd52-4ea4-82c9-1fddd6c03397 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.667016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9af72-cd52-4ea4-82c9-1fddd6c03397 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.667971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9af72-cd52-4ea4-82c9-1fddd6c03397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.670033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9af72-cd52-4ea4-82c9-1fddd6c03397 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.681342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 961/1933] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:15.685989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ad65f-5a32-46ae-a8ef-151960e66b30 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.697524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ad65f-5a32-46ae-a8ef-151960e66b30 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.703505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.704056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.704347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.704633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] PATH_INFO: `/v3/projects/ddcdf331c2974c37a4c248bbc04912f7/users/6a9ba88b730f481796109d38ba786e05/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.708742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:15.726626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddcdf331c2974c37a4c248bbc04912f7, user_id=6a9ba88b730f481796109d38ba786e05, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.729508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:15.745798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 973/1934] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:15 2022] PUT /identity/v3/projects/ddcdf331c2974c37a4c248bbc04912f7/users/6a9ba88b730f481796109d38ba786e05/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:15.757823 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984848ea-478c-49f2-b6e8-fa039c7cd965 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.757823 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984848ea-478c-49f2-b6e8-fa039c7cd965 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.758153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984848ea-478c-49f2-b6e8-fa039c7cd965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.785428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-984848ea-478c-49f2-b6e8-fa039c7cd965 None None] MFA Rules not processed for user `6a9ba88b730f481796109d38ba786e05`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:15.786674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-984848ea-478c-49f2-b6e8-fa039c7cd965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.868999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 962/1935] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:15.880756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.883173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.915787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984848ea-478c-49f2-b6e8-fa039c7cd965 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:15.918035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-984848ea-478c-49f2-b6e8-fa039c7cd965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:15.947895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:15.965515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854f5ff-9544-4545-a9e7-0542df497043 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:15.970823 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90950-2d0d-44ea-bece-f973ec307aee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:15.970823 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90950-2d0d-44ea-bece-f973ec307aee service glance] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:15.971095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90950-2d0d-44ea-bece-f973ec307aee service glance] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:15.971365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90950-2d0d-44ea-bece-f973ec307aee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:15.983347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da90950-2d0d-44ea-bece-f973ec307aee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:15.994959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da90950-2d0d-44ea-bece-f973ec307aee service glance] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:16.017596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 974/1936] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:16.020141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984848ea-478c-49f2-b6e8-fa039c7cd965 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:16.032010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:16.033105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:16.033857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:16.034414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] PATH_INFO: `/v3/projects/7020a48c951347c6ae1d177b5e5722c6` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:16.036834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7020a48c951347c6ae1d177b5e5722c6)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:16.039529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:16.101382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7020a48c951347c6ae1d177b5e5722c6', 'request_id': 'req-090eff8a-744d-4b14-99cb-bdc6de24216e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.109130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7020a48c951347c6ae1d177b5e5722c6', 'request_id': 'req-090eff8a-744d-4b14-99cb-bdc6de24216e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.119004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7020a48c951347c6ae1d177b5e5722c6', 'request_id': 'req-090eff8a-744d-4b14-99cb-bdc6de24216e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.130915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] The token cache is being invalidate because project 7020a48c951347c6ae1d177b5e5722c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:16.131318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.132648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.134332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 963/1937] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:15 2022] DELETE /identity/v3/projects/7020a48c951347c6ae1d177b5e5722c6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:16.138851 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90950-2d0d-44ea-bece-f973ec307aee service glance] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:16.152355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90950-2d0d-44ea-bece-f973ec307aee service glance] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:16.239668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:16.240584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:16.240871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:16.241148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] PATH_INFO: `/v3/users/e563a5bcfb9d40939ffbf53596f45f66` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:16.242446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e563a5bcfb9d40939ffbf53596f45f66)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:16.244556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:16.293056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e563a5bcfb9d40939ffbf53596f45f66'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.306331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e563a5bcfb9d40939ffbf53596f45f66'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.314752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e563a5bcfb9d40939ffbf53596f45f66'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.315466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e563a5bcfb9d40939ffbf53596f45f66'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.323891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6a92195d-5315-4ecf-8a85-315f4d8f9c06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e563a5bcfb9d40939ffbf53596f45f66'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.330760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 975/1938] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:16 2022] DELETE /identity/v3/users/e563a5bcfb9d40939ffbf53596f45f66 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:16.636211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:16.637961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:16.653634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090eff8a-744d-4b14-99cb-bdc6de24216e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:16.712653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:16.713554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:16.713902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:16.714236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] PATH_INFO: `/v3/projects/e328df768c58498581c53a9d3f766a6a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:16.715579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e328df768c58498581c53a9d3f766a6a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:16.717612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:16.775552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e328df768c58498581c53a9d3f766a6a', 'request_id': 'req-54ccf88c-6b28-4aff-889b-42f751090a63'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.781809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e328df768c58498581c53a9d3f766a6a', 'request_id': 'req-54ccf88c-6b28-4aff-889b-42f751090a63'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.783603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:16.790999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e328df768c58498581c53a9d3f766a6a', 'request_id': 'req-54ccf88c-6b28-4aff-889b-42f751090a63'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.805066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] The token cache is being invalidate because project e328df768c58498581c53a9d3f766a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:16.805352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.806026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-54ccf88c-6b28-4aff-889b-42f751090a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:16.807605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 964/1939] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:16 2022] DELETE /identity/v3/projects/e328df768c58498581c53a9d3f766a6a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:16.812647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd659-9eed-4feb-8759-3b2709826c6a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:16.812972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd659-9eed-4feb-8759-3b2709826c6a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:16.813242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67bd659-9eed-4feb-8759-3b2709826c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:16.826737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:16.836775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a67bd659-9eed-4feb-8759-3b2709826c6a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:16.839027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a67bd659-9eed-4feb-8759-3b2709826c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:16.901572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 976/1940] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:16.909638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a087f-ed60-4cb2-875e-f31750588be2 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:16.910013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a087f-ed60-4cb2-875e-f31750588be2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:16.910869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a087f-ed60-4cb2-875e-f31750588be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:16.931331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-4a6a087f-ed60-4cb2-875e-f31750588be2 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:16.932674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-4a6a087f-ed60-4cb2-875e-f31750588be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:16.941475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 965/1941] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:16.946478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bd659-9eed-4feb-8759-3b2709826c6a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:16.957608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67bd659-9eed-4feb-8759-3b2709826c6a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:16.963111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:16.963811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:16.964087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:16.964368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:16.965132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:16.965436 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:16.967309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:16.980468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 977/1942] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:54:16.985123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a087f-ed60-4cb2-875e-f31750588be2 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:16.996423 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a087f-ed60-4cb2-875e-f31750588be2 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.001619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bbd57-a049-4ead-81c7-c61155790704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.002396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bbd57-a049-4ead-81c7-c61155790704 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.002904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bbd57-a049-4ead-81c7-c61155790704 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.003262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bbd57-a049-4ead-81c7-c61155790704 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.004234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bbd57-a049-4ead-81c7-c61155790704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.006361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bbd57-a049-4ead-81c7-c61155790704 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.042432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 966/1943] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:16 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:17.046502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.060518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcc57ee-b431-49ea-a335-46b8f0fb50a0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.131585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cbed0-45d8-4661-aace-ff130b78ed5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.132225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cbed0-45d8-4661-aace-ff130b78ed5e admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.132504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cbed0-45d8-4661-aace-ff130b78ed5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.132781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cbed0-45d8-4661-aace-ff130b78ed5e admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.133502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cbed0-45d8-4661-aace-ff130b78ed5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.135643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cbed0-45d8-4661-aace-ff130b78ed5e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.159267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 978/1944] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:17 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:17.164110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bbd57-a049-4ead-81c7-c61155790704 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.175016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bbd57-a049-4ead-81c7-c61155790704 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.180406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b479501-9624-4287-83c8-f66a50e1aece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.181243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b479501-9624-4287-83c8-f66a50e1aece admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.181526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b479501-9624-4287-83c8-f66a50e1aece admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.181849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b479501-9624-4287-83c8-f66a50e1aece admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.182586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b479501-9624-4287-83c8-f66a50e1aece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.184196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b479501-9624-4287-83c8-f66a50e1aece admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.197978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 967/1945] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:17.203809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cbed0-45d8-4661-aace-ff130b78ed5e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.213591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cbed0-45d8-4661-aace-ff130b78ed5e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.219193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e01f84-1adc-48b1-a1e2-b5dc37d03c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.227004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01f84-1adc-48b1-a1e2-b5dc37d03c0e admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.227529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01f84-1adc-48b1-a1e2-b5dc37d03c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.228091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e01f84-1adc-48b1-a1e2-b5dc37d03c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.229391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e01f84-1adc-48b1-a1e2-b5dc37d03c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.239951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e01f84-1adc-48b1-a1e2-b5dc37d03c0e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.256737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 979/1946] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:17.261010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b479501-9624-4287-83c8-f66a50e1aece admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.271488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b479501-9624-4287-83c8-f66a50e1aece admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.276659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58554127-c938-4c9a-a25f-b8e0fc760fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.277554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58554127-c938-4c9a-a25f-b8e0fc760fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.277864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58554127-c938-4c9a-a25f-b8e0fc760fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.278140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58554127-c938-4c9a-a25f-b8e0fc760fbc admin admin] PATH_INFO: `/v3/projects/52cee473091c449e926c5ffb13158c53/users/a9ab5d3675f140a6a1878c7ccbc63026/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.293848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58554127-c938-4c9a-a25f-b8e0fc760fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=52cee473091c449e926c5ffb13158c53, user_id=a9ab5d3675f140a6a1878c7ccbc63026, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.296507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58554127-c938-4c9a-a25f-b8e0fc760fbc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.315037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 968/1947] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:17 2022] PUT /identity/v3/projects/52cee473091c449e926c5ffb13158c53/users/a9ab5d3675f140a6a1878c7ccbc63026/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:17.327618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891a14d-e8a4-44d0-9f74-a06e803caccd None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.327983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891a14d-e8a4-44d0-9f74-a06e803caccd None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.328185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891a14d-e8a4-44d0-9f74-a06e803caccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.360412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1891a14d-e8a4-44d0-9f74-a06e803caccd None None] MFA Rules not processed for user `a9ab5d3675f140a6a1878c7ccbc63026`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:17.361731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1891a14d-e8a4-44d0-9f74-a06e803caccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:17.426954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 980/1948] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:17.432152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58554127-c938-4c9a-a25f-b8e0fc760fbc admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.501915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58554127-c938-4c9a-a25f-b8e0fc760fbc admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.507636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592959c6-4905-4307-b7bf-0bc161fde714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.508444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592959c6-4905-4307-b7bf-0bc161fde714 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.508719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592959c6-4905-4307-b7bf-0bc161fde714 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.508987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592959c6-4905-4307-b7bf-0bc161fde714 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.509678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592959c6-4905-4307-b7bf-0bc161fde714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.511744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592959c6-4905-4307-b7bf-0bc161fde714 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.550331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 969/1949] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:17 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:17.552002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891a14d-e8a4-44d0-9f74-a06e803caccd None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.557556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:17.565005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891a14d-e8a4-44d0-9f74-a06e803caccd None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.623888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8008a9-c486-46ff-a00f-91029bad0cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.624595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8008a9-c486-46ff-a00f-91029bad0cbd admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.624875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8008a9-c486-46ff-a00f-91029bad0cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.625174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8008a9-c486-46ff-a00f-91029bad0cbd admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.625884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8008a9-c486-46ff-a00f-91029bad0cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.627804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8008a9-c486-46ff-a00f-91029bad0cbd admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.653527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 981/1950] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:17 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:17.654862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592959c6-4905-4307-b7bf-0bc161fde714 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.667271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592959c6-4905-4307-b7bf-0bc161fde714 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.672511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb496-6c7b-43a7-976a-49309031a7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.673176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb496-6c7b-43a7-976a-49309031a7bd admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.673465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb496-6c7b-43a7-976a-49309031a7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.673732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb496-6c7b-43a7-976a-49309031a7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.674494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdb496-6c7b-43a7-976a-49309031a7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.676219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdb496-6c7b-43a7-976a-49309031a7bd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.692084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 970/1951] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:17.694106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8008a9-c486-46ff-a00f-91029bad0cbd admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.711899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8008a9-c486-46ff-a00f-91029bad0cbd admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.720594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aad033-2ba1-4879-9ad8-b4a93b4459af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.721504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad033-2ba1-4879-9ad8-b4a93b4459af admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.721815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad033-2ba1-4879-9ad8-b4a93b4459af admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.722100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aad033-2ba1-4879-9ad8-b4a93b4459af admin admin] PATH_INFO: `/v3/projects/412240fc49614b96a0c49d2016bad2c2/users/edc872a7422340e78a35ea5e04c8c0c6/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.746943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aad033-2ba1-4879-9ad8-b4a93b4459af admin admin] RBAC: Authorizing `identity:create_grant(project_id=412240fc49614b96a0c49d2016bad2c2, user_id=edc872a7422340e78a35ea5e04c8c0c6, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.750280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aad033-2ba1-4879-9ad8-b4a93b4459af admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.775312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 982/1952] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:17 2022] PUT /identity/v3/projects/412240fc49614b96a0c49d2016bad2c2/users/edc872a7422340e78a35ea5e04c8c0c6/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:17.776320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb496-6c7b-43a7-976a-49309031a7bd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.787071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb496-6c7b-43a7-976a-49309031a7bd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.868452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:17.886003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09f2ab-2bce-4708-b832-b12be7c92d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.887466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f2ab-2bce-4708-b832-b12be7c92d80 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.887749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f2ab-2bce-4708-b832-b12be7c92d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.889864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f2ab-2bce-4708-b832-b12be7c92d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.889864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09f2ab-2bce-4708-b832-b12be7c92d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.891397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09f2ab-2bce-4708-b832-b12be7c92d80 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.905890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 971/1953] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:54:17.909682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aad033-2ba1-4879-9ad8-b4a93b4459af admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:17.922785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aad033-2ba1-4879-9ad8-b4a93b4459af admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:17.928738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:17.929447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.929851 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.930361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] PATH_INFO: `/v3/projects/412240fc49614b96a0c49d2016bad2c2/users/edc872a7422340e78a35ea5e04c8c0c6/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:17.932314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] RBAC: Authorizing `identity:create_grant(project_id=412240fc49614b96a0c49d2016bad2c2, user_id=edc872a7422340e78a35ea5e04c8c0c6, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:17.935124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:17.951432 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 983/1954] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:17 2022] PUT /identity/v3/projects/412240fc49614b96a0c49d2016bad2c2/users/edc872a7422340e78a35ea5e04c8c0c6/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:17.962198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1005-63df-420b-a95a-51e72ba119fe None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:17.962576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1005-63df-420b-a95a-51e72ba119fe None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:17.963066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1005-63df-420b-a95a-51e72ba119fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:18.004534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-495c1005-63df-420b-a95a-51e72ba119fe None None] MFA Rules not processed for user `edc872a7422340e78a35ea5e04c8c0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:18.006377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-495c1005-63df-420b-a95a-51e72ba119fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:18.130115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:18.132670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:18.185876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 972/1955] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:18.201461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c1005-63df-420b-a95a-51e72ba119fe None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:18.206695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-495c1005-63df-420b-a95a-51e72ba119fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:18.260204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190f148-cc69-4969-9cda-242040ec751a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:18.269932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:18.271047 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:18.271333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:18.271608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:18.277236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:18.296275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c1005-63df-420b-a95a-51e72ba119fe None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:18.301379 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769a04e-8f89-4e47-bad2-43a45df2457e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:18.302207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769a04e-8f89-4e47-bad2-43a45df2457e service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:18.302401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769a04e-8f89-4e47-bad2-43a45df2457e service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:18.303358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769a04e-8f89-4e47-bad2-43a45df2457e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:18.323568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769a04e-8f89-4e47-bad2-43a45df2457e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:18.327430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769a04e-8f89-4e47-bad2-43a45df2457e service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:18.356976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 973/1956] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:18.424952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:18.427103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:18.442203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 984/1957] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:18.728722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769a04e-8f89-4e47-bad2-43a45df2457e service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:18.742956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769a04e-8f89-4e47-bad2-43a45df2457e service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:18.747529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae1d0f-9bc8-4432-92c6-a6962ccadedf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:18.748678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1d0f-9bc8-4432-92c6-a6962ccadedf service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:18.749153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1d0f-9bc8-4432-92c6-a6962ccadedf service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:18.750231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae1d0f-9bc8-4432-92c6-a6962ccadedf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:18.752921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-54ae1d0f-9bc8-4432-92c6-a6962ccadedf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:18.822122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae1d0f-9bc8-4432-92c6-a6962ccadedf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:18.823833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae1d0f-9bc8-4432-92c6-a6962ccadedf service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:18.834487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 974/1958] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:19.207546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:19.214854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:19.289380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8617cf0-7da9-4a1f-9ef3-b0f25a5b8638 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:19.296246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:19.297149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:19.297434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:19.297704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] PATH_INFO: `/v3/users/a9ab5d3675f140a6a1878c7ccbc63026` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:19.299517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ab5d3675f140a6a1878c7ccbc63026)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:19.302306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:19.358845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ab5d3675f140a6a1878c7ccbc63026'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:19.375735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ab5d3675f140a6a1878c7ccbc63026'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:19.391696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ab5d3675f140a6a1878c7ccbc63026'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:19.392394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ab5d3675f140a6a1878c7ccbc63026'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:19.401682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ab5d3675f140a6a1878c7ccbc63026'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:19.421510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 985/1959] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:19 2022] DELETE /identity/v3/users/a9ab5d3675f140a6a1878c7ccbc63026 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:19.426410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917dd22-4bbb-4e20-93f2-9a99f620a33e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:19.426703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917dd22-4bbb-4e20-93f2-9a99f620a33e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:19.426984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917dd22-4bbb-4e20-93f2-9a99f620a33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:19.456240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a917dd22-4bbb-4e20-93f2-9a99f620a33e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:19.458548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a917dd22-4bbb-4e20-93f2-9a99f620a33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:19.472990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:19.575499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 975/1960] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:19.585983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:19.587612 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:19.657405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a90395-b226-416f-b124-083158bc9a81 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:19.662721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa950262-1a97-410d-86b5-fcefd8ae4aa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:19.663382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa950262-1a97-410d-86b5-fcefd8ae4aa3 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:19.663822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa950262-1a97-410d-86b5-fcefd8ae4aa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:19.664275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa950262-1a97-410d-86b5-fcefd8ae4aa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:19.674543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa950262-1a97-410d-86b5-fcefd8ae4aa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:19.679147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa950262-1a97-410d-86b5-fcefd8ae4aa3 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:19.703184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 986/1961] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:20.002686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917dd22-4bbb-4e20-93f2-9a99f620a33e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:20.006209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a917dd22-4bbb-4e20-93f2-9a99f620a33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:20.018845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917dd22-4bbb-4e20-93f2-9a99f620a33e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:20.025895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:20.027033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:20.027506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:20.027788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] PATH_INFO: `/v3/projects/52cee473091c449e926c5ffb13158c53` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:20.029922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=52cee473091c449e926c5ffb13158c53)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:20.031867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:20.084760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52cee473091c449e926c5ffb13158c53', 'request_id': 'req-f8ce5893-c796-4c3b-9a26-da2bf430a05e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.091564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52cee473091c449e926c5ffb13158c53', 'request_id': 'req-f8ce5893-c796-4c3b-9a26-da2bf430a05e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.101075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52cee473091c449e926c5ffb13158c53', 'request_id': 'req-f8ce5893-c796-4c3b-9a26-da2bf430a05e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.112661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] The token cache is being invalidate because project 52cee473091c449e926c5ffb13158c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:20.113391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.114104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.115796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 976/1962] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:20 2022] DELETE /identity/v3/projects/52cee473091c449e926c5ffb13158c53 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:20.120356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa950262-1a97-410d-86b5-fcefd8ae4aa3 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:20.133434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa950262-1a97-410d-86b5-fcefd8ae4aa3 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:20.160350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:20.201343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:20.202255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:20.202542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:20.202818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] PATH_INFO: `/v3/users/edc872a7422340e78a35ea5e04c8c0c6` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:20.204039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc872a7422340e78a35ea5e04c8c0c6)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:20.205997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:20.259476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc872a7422340e78a35ea5e04c8c0c6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.271483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc872a7422340e78a35ea5e04c8c0c6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.279875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc872a7422340e78a35ea5e04c8c0c6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.280493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc872a7422340e78a35ea5e04c8c0c6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.287942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f29c641e-3f1e-4f69-af15-c91274be6f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc872a7422340e78a35ea5e04c8c0c6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.293936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 987/1963] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:20 2022] DELETE /identity/v3/users/edc872a7422340e78a35ea5e04c8c0c6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:20.587667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:20.589564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:20.603638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce5893-c796-4c3b-9a26-da2bf430a05e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:20.665519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:20.666440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:20.666718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:20.666990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] PATH_INFO: `/v3/projects/412240fc49614b96a0c49d2016bad2c2` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:20.669608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=412240fc49614b96a0c49d2016bad2c2)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:20.672703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:20.734884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412240fc49614b96a0c49d2016bad2c2', 'request_id': 'req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.739244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:20.743795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412240fc49614b96a0c49d2016bad2c2', 'request_id': 'req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.756071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412240fc49614b96a0c49d2016bad2c2', 'request_id': 'req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.784463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] The token cache is being invalidate because project 412240fc49614b96a0c49d2016bad2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:20.784807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.785656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6f3ebe5b-3d57-43d1-8398-9979cfdc6111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:20.792687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 977/1964] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:20 2022] DELETE /identity/v3/projects/412240fc49614b96a0c49d2016bad2c2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:20.797985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cc87e-55f9-41bc-ae4c-fe8cbeaa747f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:20.798378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cc87e-55f9-41bc-ae4c-fe8cbeaa747f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:20.798685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cc87e-55f9-41bc-ae4c-fe8cbeaa747f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:20.819225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-9e7cc87e-55f9-41bc-ae4c-fe8cbeaa747f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:20.820527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9e7cc87e-55f9-41bc-ae4c-fe8cbeaa747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:20.881800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 988/1965] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:20.889725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5811db-3733-4393-ac7a-96e3fe51ce8a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:20.890046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5811db-3733-4393-ac7a-96e3fe51ce8a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:20.890321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5811db-3733-4393-ac7a-96e3fe51ce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:20.911286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-da5811db-3733-4393-ac7a-96e3fe51ce8a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:20.912500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-da5811db-3733-4393-ac7a-96e3fe51ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:20.921551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 978/1966] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:20.926344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cc87e-55f9-41bc-ae4c-fe8cbeaa747f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:20.936088 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cc87e-55f9-41bc-ae4c-fe8cbeaa747f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:20.941030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:20.941811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:20.942109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:20.942388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:20.943130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:20.943438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:20.945063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:20.957347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 989/1967] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:54:20.961550 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5811db-3733-4393-ac7a-96e3fe51ce8a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:20.972244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5811db-3733-4393-ac7a-96e3fe51ce8a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:20.977633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be2426-2a5e-4b3b-a94b-e9a1cf06f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:20.978360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be2426-2a5e-4b3b-a94b-e9a1cf06f5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:20.978654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be2426-2a5e-4b3b-a94b-e9a1cf06f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:20.978954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be2426-2a5e-4b3b-a94b-e9a1cf06f5c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:20.979670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be2426-2a5e-4b3b-a94b-e9a1cf06f5c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:20.982760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be2426-2a5e-4b3b-a94b-e9a1cf06f5c0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:21.017937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 979/1968] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:20 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:21.022175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:21.032721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbb193-6972-425e-ac27-c09049deb2bb admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:21.066886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:21.101967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508ae44-c285-42fc-86d3-32dfb816716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:21.103090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508ae44-c285-42fc-86d3-32dfb816716a admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:21.103561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508ae44-c285-42fc-86d3-32dfb816716a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:21.104034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508ae44-c285-42fc-86d3-32dfb816716a admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:21.105254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508ae44-c285-42fc-86d3-32dfb816716a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:21.107833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508ae44-c285-42fc-86d3-32dfb816716a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:21.132907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 990/1969] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:21 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:21.137046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be2426-2a5e-4b3b-a94b-e9a1cf06f5c0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:21.149277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be2426-2a5e-4b3b-a94b-e9a1cf06f5c0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:21.155332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1748f1-a5ec-4558-bfd4-2fd2715ffacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:21.156120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1748f1-a5ec-4558-bfd4-2fd2715ffacb admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:21.156450 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1748f1-a5ec-4558-bfd4-2fd2715ffacb admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:21.156790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1748f1-a5ec-4558-bfd4-2fd2715ffacb admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:21.157621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1748f1-a5ec-4558-bfd4-2fd2715ffacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:21.159336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1748f1-a5ec-4558-bfd4-2fd2715ffacb admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:21.171630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 980/1970] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:21.176342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508ae44-c285-42fc-86d3-32dfb816716a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:21.190231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508ae44-c285-42fc-86d3-32dfb816716a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:21.196762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6cf4b-753d-4cd3-9a5f-fe11bf0ad4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:21.197454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6cf4b-753d-4cd3-9a5f-fe11bf0ad4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:21.197734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6cf4b-753d-4cd3-9a5f-fe11bf0ad4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:21.198058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6cf4b-753d-4cd3-9a5f-fe11bf0ad4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:21.198809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6cf4b-753d-4cd3-9a5f-fe11bf0ad4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:21.200420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6cf4b-753d-4cd3-9a5f-fe11bf0ad4d6 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:21.216060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 991/1971] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:21.220775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1748f1-a5ec-4558-bfd4-2fd2715ffacb admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:21.232003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1748f1-a5ec-4558-bfd4-2fd2715ffacb admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:21.239683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:21.240574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:21.240847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:21.241120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] PATH_INFO: `/v3/projects/95e4165ff5ac4ddb9174ee57a855261a/users/d0e648d49aaf4aafb296f6f9dc9b927f/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:21.258783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e4165ff5ac4ddb9174ee57a855261a, user_id=d0e648d49aaf4aafb296f6f9dc9b927f, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:21.261414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:21.279168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 981/1972] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:21 2022] PUT /identity/v3/projects/95e4165ff5ac4ddb9174ee57a855261a/users/d0e648d49aaf4aafb296f6f9dc9b927f/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:21.291341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdceee-0d7c-4fcd-b1c7-73892047577c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:21.291664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdceee-0d7c-4fcd-b1c7-73892047577c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:21.291969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdceee-0d7c-4fcd-b1c7-73892047577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:21.327243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-30cdceee-0d7c-4fcd-b1c7-73892047577c None None] MFA Rules not processed for user `d0e648d49aaf4aafb296f6f9dc9b927f`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:21.328539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-30cdceee-0d7c-4fcd-b1c7-73892047577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:21.400749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 992/1973] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:21.417317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:21.420239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:21.486555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399f149-9b76-4725-86f6-75b8c244e325 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:21.491698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3afad4-7311-4174-b00f-d601bd5d770a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:21.492343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3afad4-7311-4174-b00f-d601bd5d770a service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:21.492605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3afad4-7311-4174-b00f-d601bd5d770a service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:21.492882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3afad4-7311-4174-b00f-d601bd5d770a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:21.502997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3afad4-7311-4174-b00f-d601bd5d770a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:21.504586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3afad4-7311-4174-b00f-d601bd5d770a service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:21.514088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 982/1974] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:26.553043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdceee-0d7c-4fcd-b1c7-73892047577c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:26.554551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-30cdceee-0d7c-4fcd-b1c7-73892047577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:26.642422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdceee-0d7c-4fcd-b1c7-73892047577c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:26.647740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:26.648635 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:26.648913 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:26.649185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] PATH_INFO: `/v3/users/5dd369eba6a64bba8651983ed9e28d78` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:26.650402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd369eba6a64bba8651983ed9e28d78)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:26.652499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:26.702759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:26.705520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd369eba6a64bba8651983ed9e28d78'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:26.725943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd369eba6a64bba8651983ed9e28d78'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:26.734850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd369eba6a64bba8651983ed9e28d78'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:26.735487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd369eba6a64bba8651983ed9e28d78'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:26.743648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd369eba6a64bba8651983ed9e28d78'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:26.753317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 993/1975] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:26 2022] DELETE /identity/v3/users/5dd369eba6a64bba8651983ed9e28d78 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:26.755443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68e5c-695d-47a4-aed6-b4d6cf5037af None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:26.755946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68e5c-695d-47a4-aed6-b4d6cf5037af None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:26.756078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68e5c-695d-47a4-aed6-b4d6cf5037af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:26.775111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2ca68e5c-695d-47a4-aed6-b4d6cf5037af None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:26.776428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2ca68e5c-695d-47a4-aed6-b4d6cf5037af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:26.853137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 983/1976] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:26.856879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:26.858658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:26.924313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6233e3c-c8da-4f68-9765-45eebe889205 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:26.928461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:26.929122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:26.929405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:26.929683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:26.939542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:26.941057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:26.950933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 994/1977] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:27.326188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca68e5c-695d-47a4-aed6-b4d6cf5037af None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:27.327259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2ca68e5c-695d-47a4-aed6-b4d6cf5037af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:27.340073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca68e5c-695d-47a4-aed6-b4d6cf5037af None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:27.345534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:27.346515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:27.346832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:27.347137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] PATH_INFO: `/v3/users/d0e648d49aaf4aafb296f6f9dc9b927f` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:27.348460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e648d49aaf4aafb296f6f9dc9b927f)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:27.352446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:27.353372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:27.421801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:27.424385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:27.441501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b54cc-5a9e-4a19-beeb-120b5084e5f4 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:27.444779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e648d49aaf4aafb296f6f9dc9b927f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.447211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:27.448090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:27.448382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:27.448661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] PATH_INFO: `/v3/projects/c749343f39de4722999ca748ad05be32` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:27.449906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c749343f39de4722999ca748ad05be32)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:27.451896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:27.467642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e648d49aaf4aafb296f6f9dc9b927f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.477057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e648d49aaf4aafb296f6f9dc9b927f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.485905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e648d49aaf4aafb296f6f9dc9b927f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.486817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b54d4c6f-adbf-4365-8e8a-a050a7a43818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e648d49aaf4aafb296f6f9dc9b927f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.495804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 984/1978] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:27 2022] DELETE /identity/v3/users/d0e648d49aaf4aafb296f6f9dc9b927f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:27.500220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d832dd-3349-4d3c-a85c-b17b95efd029 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:27.500599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d832dd-3349-4d3c-a85c-b17b95efd029 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:27.500968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d832dd-3349-4d3c-a85c-b17b95efd029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:27.505416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c749343f39de4722999ca748ad05be32', 'request_id': 'req-16129fa8-d200-40ba-9c7f-cdcf5de8417d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.515226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c749343f39de4722999ca748ad05be32', 'request_id': 'req-16129fa8-d200-40ba-9c7f-cdcf5de8417d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.520463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-23d832dd-3349-4d3c-a85c-b17b95efd029 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:27.521753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-23d832dd-3349-4d3c-a85c-b17b95efd029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:27.522868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c749343f39de4722999ca748ad05be32', 'request_id': 'req-16129fa8-d200-40ba-9c7f-cdcf5de8417d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.536417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] The token cache is being invalidate because project c749343f39de4722999ca748ad05be32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:27.536721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.537535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.539848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 995/1979] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:27 2022] DELETE /identity/v3/projects/c749343f39de4722999ca748ad05be32 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:27.544127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:27.545618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:27.594123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 985/1980] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:27.600081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:27.605044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d832dd-3349-4d3c-a85c-b17b95efd029 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:27.606537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-23d832dd-3349-4d3c-a85c-b17b95efd029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:27.619137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16129fa8-d200-40ba-9c7f-cdcf5de8417d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:27.630788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:27.631547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:27.631836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:27.632101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] PATH_INFO: `/v3/users/92fa886675704f708a3ca9f2dbb549b4` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:27.633346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92fa886675704f708a3ca9f2dbb549b4)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:27.635354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:27.677428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d832dd-3349-4d3c-a85c-b17b95efd029 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:27.683117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e649ec-1f28-439e-8323-be08f9ce3d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:27.683929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e649ec-1f28-439e-8323-be08f9ce3d6c service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:27.684283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e649ec-1f28-439e-8323-be08f9ce3d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:27.684569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e649ec-1f28-439e-8323-be08f9ce3d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:27.691768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92fa886675704f708a3ca9f2dbb549b4'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.697278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e649ec-1f28-439e-8323-be08f9ce3d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:27.698874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e649ec-1f28-439e-8323-be08f9ce3d6c service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:27.709239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92fa886675704f708a3ca9f2dbb549b4'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.710924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 986/1981] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:27.719879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92fa886675704f708a3ca9f2dbb549b4'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.721265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92fa886675704f708a3ca9f2dbb549b4'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.735965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92fa886675704f708a3ca9f2dbb549b4'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:27.747936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 996/1982] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:27 2022] DELETE /identity/v3/users/92fa886675704f708a3ca9f2dbb549b4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:28.108112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e649ec-1f28-439e-8323-be08f9ce3d6c service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.118990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e649ec-1f28-439e-8323-be08f9ce3d6c service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.143177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:28.187141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.187726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.188803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.189088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.189362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] PATH_INFO: `/v3/projects/6692fa0997f8426eadb3fc5302626234` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.191118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:28.192058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6692fa0997f8426eadb3fc5302626234)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.194044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.210941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28363d6-525e-441b-8592-fea1539564d5 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.218398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.219628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.219929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.220208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] PATH_INFO: `/v3/projects/95e4165ff5ac4ddb9174ee57a855261a` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.221648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e4165ff5ac4ddb9174ee57a855261a)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.224860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.266480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6692fa0997f8426eadb3fc5302626234', 'request_id': 'req-c4e08850-75af-4c43-87ad-185a12b75761'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.272974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6692fa0997f8426eadb3fc5302626234', 'request_id': 'req-c4e08850-75af-4c43-87ad-185a12b75761'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.282183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6692fa0997f8426eadb3fc5302626234', 'request_id': 'req-c4e08850-75af-4c43-87ad-185a12b75761'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.286749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e4165ff5ac4ddb9174ee57a855261a', 'request_id': 'req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.293989 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] The token cache is being invalidate because project 6692fa0997f8426eadb3fc5302626234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:28.294274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.294384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e4165ff5ac4ddb9174ee57a855261a', 'request_id': 'req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.294902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.296509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 987/1983] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:28 2022] DELETE /identity/v3/projects/6692fa0997f8426eadb3fc5302626234 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:28.300620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.302000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:28.302980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e4165ff5ac4ddb9174ee57a855261a', 'request_id': 'req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.314126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] The token cache is being invalidate because project 95e4165ff5ac4ddb9174ee57a855261a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:28.314441 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.315159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9f1e9688-f046-4c9e-8ed8-d03f96b8b7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.317617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 997/1984] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:28 2022] DELETE /identity/v3/projects/95e4165ff5ac4ddb9174ee57a855261a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:28.323051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6658a-3540-46f5-a7a8-791b3ad391b6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.323410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6658a-3540-46f5-a7a8-791b3ad391b6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.323755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6658a-3540-46f5-a7a8-791b3ad391b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.342333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-14d6658a-3540-46f5-a7a8-791b3ad391b6 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:28.343532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-14d6658a-3540-46f5-a7a8-791b3ad391b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:28.369413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e08850-75af-4c43-87ad-185a12b75761 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.388509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:28.405495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 998/1985] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:28.413161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed29187-222b-4164-9140-1ea4c3f08f56 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.413454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed29187-222b-4164-9140-1ea4c3f08f56 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.413795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed29187-222b-4164-9140-1ea4c3f08f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.430249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.431147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.431431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.431704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] PATH_INFO: `/v3/users/29ef1ef69cbe481c949431154655eecd` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.432935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ef1ef69cbe481c949431154655eecd)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.433281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6ed29187-222b-4164-9140-1ea4c3f08f56 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:28.434540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6ed29187-222b-4164-9140-1ea4c3f08f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:28.434958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.438088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:28.447652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 999/1986] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:28.448773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed29187-222b-4164-9140-1ea4c3f08f56 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.459039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed29187-222b-4164-9140-1ea4c3f08f56 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.465250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.465250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.465250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.465250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.465989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:28.466324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.468265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.484883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1000/1987] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:54:28.489576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.499301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ef1ef69cbe481c949431154655eecd'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.507964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5834979-860a-4e49-80c7-ff5d2cbdbff6 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.515177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f5fd-0dc6-4a76-bf79-7202687cedf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.516099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f5fd-0dc6-4a76-bf79-7202687cedf4 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.516679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ef1ef69cbe481c949431154655eecd'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.516875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f5fd-0dc6-4a76-bf79-7202687cedf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.517166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f5fd-0dc6-4a76-bf79-7202687cedf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.517903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845f5fd-0dc6-4a76-bf79-7202687cedf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.520155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845f5fd-0dc6-4a76-bf79-7202687cedf4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.526732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ef1ef69cbe481c949431154655eecd'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.536427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ef1ef69cbe481c949431154655eecd'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.537287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ef1ef69cbe481c949431154655eecd'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.548506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 988/1988] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:28 2022] DELETE /identity/v3/users/29ef1ef69cbe481c949431154655eecd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:28.552622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1001/1989] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:28 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:54:28.559028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.560702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:28.640965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46fdfc-256e-449f-840f-946d4ea2a693 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.721613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de588b43-cb84-40e4-90cc-6af08ca45589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.723098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de588b43-cb84-40e4-90cc-6af08ca45589 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.723098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de588b43-cb84-40e4-90cc-6af08ca45589 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.723330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de588b43-cb84-40e4-90cc-6af08ca45589 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.723929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de588b43-cb84-40e4-90cc-6af08ca45589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.726145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de588b43-cb84-40e4-90cc-6af08ca45589 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.749815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 989/1990] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:28 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:28.752970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f5fd-0dc6-4a76-bf79-7202687cedf4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.763940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f5fd-0dc6-4a76-bf79-7202687cedf4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.768908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123932dd-3e14-44b1-8d19-4c00a2aaf0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.769646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123932dd-3e14-44b1-8d19-4c00a2aaf0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.769905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123932dd-3e14-44b1-8d19-4c00a2aaf0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.770184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123932dd-3e14-44b1-8d19-4c00a2aaf0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.771331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123932dd-3e14-44b1-8d19-4c00a2aaf0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.772527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123932dd-3e14-44b1-8d19-4c00a2aaf0a3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.785725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1002/1991] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:28.790216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de588b43-cb84-40e4-90cc-6af08ca45589 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.801895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de588b43-cb84-40e4-90cc-6af08ca45589 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.809369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.810191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.810524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.810865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.811656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.813347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.824662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 990/1992] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:54:28.829654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123932dd-3e14-44b1-8d19-4c00a2aaf0a3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.840788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123932dd-3e14-44b1-8d19-4c00a2aaf0a3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.846620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff98ce-f32e-46bb-b372-2a1688b81026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.848300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff98ce-f32e-46bb-b372-2a1688b81026 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.848772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff98ce-f32e-46bb-b372-2a1688b81026 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.849556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff98ce-f32e-46bb-b372-2a1688b81026 admin admin] PATH_INFO: `/v3/projects/0a96ea38c3b64eddb3458c4a24b1f2e2/users/65b2c230e0a6451e88c3bced68ca3892/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.859234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:28.861121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:28.875166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe0344-254a-48ad-9c80-42c071027800 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:28.880369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff98ce-f32e-46bb-b372-2a1688b81026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a96ea38c3b64eddb3458c4a24b1f2e2, user_id=65b2c230e0a6451e88c3bced68ca3892, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.880546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:28.881183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.881466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.881742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] PATH_INFO: `/v3/projects/1be5c03fb3ec45558b9c70c24618247d` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.882355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff98ce-f32e-46bb-b372-2a1688b81026 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.888720 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1be5c03fb3ec45558b9c70c24618247d)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:28.890919 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:28.892892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:28.899783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1003/1993] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:28 2022] PUT /identity/v3/projects/0a96ea38c3b64eddb3458c4a24b1f2e2/users/65b2c230e0a6451e88c3bced68ca3892/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:28.912055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c916349-e56f-49e4-a18c-9e5ebdf2cea1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:28.912373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c916349-e56f-49e4-a18c-9e5ebdf2cea1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:28.912623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c916349-e56f-49e4-a18c-9e5ebdf2cea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:28.943742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3c916349-e56f-49e4-a18c-9e5ebdf2cea1 None None] MFA Rules not processed for user `65b2c230e0a6451e88c3bced68ca3892`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:28.944985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3c916349-e56f-49e4-a18c-9e5ebdf2cea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:28.954684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1be5c03fb3ec45558b9c70c24618247d', 'request_id': 'req-43318fd0-80ce-4dc8-8ee1-a3bda853f445'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.962667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1be5c03fb3ec45558b9c70c24618247d', 'request_id': 'req-43318fd0-80ce-4dc8-8ee1-a3bda853f445'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.974687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1be5c03fb3ec45558b9c70c24618247d', 'request_id': 'req-43318fd0-80ce-4dc8-8ee1-a3bda853f445'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.987655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] The token cache is being invalidate because project 1be5c03fb3ec45558b9c70c24618247d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:28.987985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.988820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-43318fd0-80ce-4dc8-8ee1-a3bda853f445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:28.991647 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 991/1994] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:28 2022] DELETE /identity/v3/projects/1be5c03fb3ec45558b9c70c24618247d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:29.008527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c20cf-d5d7-4355-8275-92d8bc648d9e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.008922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c20cf-d5d7-4355-8275-92d8bc648d9e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.009345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c20cf-d5d7-4355-8275-92d8bc648d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.042642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-994c20cf-d5d7-4355-8275-92d8bc648d9e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:29.047337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-994c20cf-d5d7-4355-8275-92d8bc648d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:29.055497 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1004/1995] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:29.068375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c916349-e56f-49e4-a18c-9e5ebdf2cea1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.070370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3c916349-e56f-49e4-a18c-9e5ebdf2cea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:29.153917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 992/1996] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:29.157793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e8369-e211-45c5-9f42-cebb83b532e4 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.158154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e8369-e211-45c5-9f42-cebb83b532e4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.158498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e8369-e211-45c5-9f42-cebb83b532e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.164154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c916349-e56f-49e4-a18c-9e5ebdf2cea1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.169007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08285c-e2ae-4bd8-801d-cfb701668583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.169659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08285c-e2ae-4bd8-801d-cfb701668583 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.169973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08285c-e2ae-4bd8-801d-cfb701668583 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.170253 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08285c-e2ae-4bd8-801d-cfb701668583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.182241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-bd7e8369-e211-45c5-9f42-cebb83b532e4 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:29.183573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bd7e8369-e211-45c5-9f42-cebb83b532e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:29.183904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08285c-e2ae-4bd8-801d-cfb701668583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.185817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08285c-e2ae-4bd8-801d-cfb701668583 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.198404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 993/1997] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:29.201676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1005/1998] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:54:29.201887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e8369-e211-45c5-9f42-cebb83b532e4 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.212979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e8369-e211-45c5-9f42-cebb83b532e4 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.217891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.218618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.218905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.219185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.219913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:29.220225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.221944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.240472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 994/1999] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:54:29.241799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08285c-e2ae-4bd8-801d-cfb701668583 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.252257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:29.257214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08285c-e2ae-4bd8-801d-cfb701668583 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.263688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a1961-fe14-4b26-bcfc-8f24f67aaca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.264492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a1961-fe14-4b26-bcfc-8f24f67aaca0 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.264849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a1961-fe14-4b26-bcfc-8f24f67aaca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.265198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a1961-fe14-4b26-bcfc-8f24f67aaca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.266034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a1961-fe14-4b26-bcfc-8f24f67aaca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.268450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a1961-fe14-4b26-bcfc-8f24f67aaca0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.320668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1006/2000] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:29 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:29.322383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.334010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ea09c-3206-49e0-8b43-febc77543eca admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.396369 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6b890-45dc-40a8-8936-6495c314dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.397065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6b890-45dc-40a8-8936-6495c314dfa7 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.397365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6b890-45dc-40a8-8936-6495c314dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.397635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6b890-45dc-40a8-8936-6495c314dfa7 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.398397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6b890-45dc-40a8-8936-6495c314dfa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.400474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6b890-45dc-40a8-8936-6495c314dfa7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.423908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:29.430066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 995/2001] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:29 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:29.434478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a1961-fe14-4b26-bcfc-8f24f67aaca0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.449126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a1961-fe14-4b26-bcfc-8f24f67aaca0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.454355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b000e-09c2-4f91-98e9-d37b6c2f375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.455033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b000e-09c2-4f91-98e9-d37b6c2f375c admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.455317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b000e-09c2-4f91-98e9-d37b6c2f375c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.455597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b000e-09c2-4f91-98e9-d37b6c2f375c admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.456760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b000e-09c2-4f91-98e9-d37b6c2f375c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.458888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b000e-09c2-4f91-98e9-d37b6c2f375c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.471486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1007/2002] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:54:29.475752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6b890-45dc-40a8-8936-6495c314dfa7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.486249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6b890-45dc-40a8-8936-6495c314dfa7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.491256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635ab5a-dad6-4d36-8b0f-12437aaa5358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.491956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ab5a-dad6-4d36-8b0f-12437aaa5358 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.492225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ab5a-dad6-4d36-8b0f-12437aaa5358 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.492503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ab5a-dad6-4d36-8b0f-12437aaa5358 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.493236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635ab5a-dad6-4d36-8b0f-12437aaa5358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.494963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635ab5a-dad6-4d36-8b0f-12437aaa5358 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.511600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 996/2003] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:29.515941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b000e-09c2-4f91-98e9-d37b6c2f375c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.531289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b000e-09c2-4f91-98e9-d37b6c2f375c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.536701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3b95-510f-42f1-bbe8-8a56f1267c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.537610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3b95-510f-42f1-bbe8-8a56f1267c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.537934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3b95-510f-42f1-bbe8-8a56f1267c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.538246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3b95-510f-42f1-bbe8-8a56f1267c8c admin admin] PATH_INFO: `/v3/projects/2716725ceb8e4e6a99c79930f7549d78/users/fafe7045c3774f38b4ddb9cdd52c4433/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.558918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab3b95-510f-42f1-bbe8-8a56f1267c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2716725ceb8e4e6a99c79930f7549d78, user_id=fafe7045c3774f38b4ddb9cdd52c4433, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.561725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab3b95-510f-42f1-bbe8-8a56f1267c8c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.577022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1008/2004] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:29 2022] PUT /identity/v3/projects/2716725ceb8e4e6a99c79930f7549d78/users/fafe7045c3774f38b4ddb9cdd52c4433/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:29.588918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf19bc1-7434-4c06-ba48-347684da20e9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.589245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf19bc1-7434-4c06-ba48-347684da20e9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.589570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf19bc1-7434-4c06-ba48-347684da20e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.620449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7bf19bc1-7434-4c06-ba48-347684da20e9 None None] MFA Rules not processed for user `fafe7045c3774f38b4ddb9cdd52c4433`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:29.621726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7bf19bc1-7434-4c06-ba48-347684da20e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:29.689486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 997/2005] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:29.694295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3b95-510f-42f1-bbe8-8a56f1267c8c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.768583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3b95-510f-42f1-bbe8-8a56f1267c8c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.774334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7935-96eb-41c6-892d-400c9bb4fff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.775262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7935-96eb-41c6-892d-400c9bb4fff7 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.775614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7935-96eb-41c6-892d-400c9bb4fff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.775956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7935-96eb-41c6-892d-400c9bb4fff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.776761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a7935-96eb-41c6-892d-400c9bb4fff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.778859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a7935-96eb-41c6-892d-400c9bb4fff7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.812323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1009/2006] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:29 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:29.816495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf19bc1-7434-4c06-ba48-347684da20e9 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.828199 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf19bc1-7434-4c06-ba48-347684da20e9 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.904880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d770a-4014-497b-8368-b77baf73261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.906110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d770a-4014-497b-8368-b77baf73261d admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.906401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d770a-4014-497b-8368-b77baf73261d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.906683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d770a-4014-497b-8368-b77baf73261d admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.907388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d770a-4014-497b-8368-b77baf73261d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.909426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d770a-4014-497b-8368-b77baf73261d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.932779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 998/2007] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:29 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:54:29.936820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7935-96eb-41c6-892d-400c9bb4fff7 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:29.947996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7935-96eb-41c6-892d-400c9bb4fff7 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:29.953754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e514-e9a9-4f9b-b70e-5a3fd8a1ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:29.955599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e514-e9a9-4f9b-b70e-5a3fd8a1ad3b admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:29.956025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e514-e9a9-4f9b-b70e-5a3fd8a1ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:29.956496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11e514-e9a9-4f9b-b70e-5a3fd8a1ad3b admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:29.957557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11e514-e9a9-4f9b-b70e-5a3fd8a1ad3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:29.959319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11e514-e9a9-4f9b-b70e-5a3fd8a1ad3b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:29.977073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1010/2008] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:54:29.981178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d770a-4014-497b-8368-b77baf73261d admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:30.004856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d770a-4014-497b-8368-b77baf73261d admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:30.010010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22d0f4-4a91-4f42-afa8-3a8711137d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:30.010927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d0f4-4a91-4f42-afa8-3a8711137d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:30.011216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d0f4-4a91-4f42-afa8-3a8711137d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:30.011496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22d0f4-4a91-4f42-afa8-3a8711137d40 admin admin] PATH_INFO: `/v3/projects/0e8ce980670840dcb79805a605be1861/users/aed451e26abf49d8946d189789049f5a/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:30.030229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22d0f4-4a91-4f42-afa8-3a8711137d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8ce980670840dcb79805a605be1861, user_id=aed451e26abf49d8946d189789049f5a, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:30.032982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22d0f4-4a91-4f42-afa8-3a8711137d40 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:30.048106 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 999/2009] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:29 2022] PUT /identity/v3/projects/0e8ce980670840dcb79805a605be1861/users/aed451e26abf49d8946d189789049f5a/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:30.052434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e514-e9a9-4f9b-b70e-5a3fd8a1ad3b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:30.063292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11e514-e9a9-4f9b-b70e-5a3fd8a1ad3b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:30.129818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c8746-61e7-48e2-bf77-1311793a2024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:30.130495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8746-61e7-48e2-bf77-1311793a2024 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:30.130776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8746-61e7-48e2-bf77-1311793a2024 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:30.131026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8746-61e7-48e2-bf77-1311793a2024 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:30.131753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c8746-61e7-48e2-bf77-1311793a2024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:30.133361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c8746-61e7-48e2-bf77-1311793a2024 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:30.144489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1011/2010] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:30.148578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22d0f4-4a91-4f42-afa8-3a8711137d40 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:30.160177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22d0f4-4a91-4f42-afa8-3a8711137d40 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:30.165206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:30.166273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:30.166560 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:30.166835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] PATH_INFO: `/v3/projects/0e8ce980670840dcb79805a605be1861/users/aed451e26abf49d8946d189789049f5a/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:30.168866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8ce980670840dcb79805a605be1861, user_id=aed451e26abf49d8946d189789049f5a, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:30.171545 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:30.182673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1000/2011] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:30 2022] PUT /identity/v3/projects/0e8ce980670840dcb79805a605be1861/users/aed451e26abf49d8946d189789049f5a/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:30.193585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554a6c-a490-432e-b138-6b3f712de546 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:30.194043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554a6c-a490-432e-b138-6b3f712de546 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:30.194499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef554a6c-a490-432e-b138-6b3f712de546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:30.222149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ef554a6c-a490-432e-b138-6b3f712de546 None None] MFA Rules not processed for user `aed451e26abf49d8946d189789049f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:30.223541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ef554a6c-a490-432e-b138-6b3f712de546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:30.228887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:30.307104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1012/2012] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:30.318577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:30.320631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:30.360615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:30.388452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efece5-16fc-4d0d-8f85-c57067a4d99e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:30.392251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73011e-552f-4e42-ba7b-908daff1367f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:30.392955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73011e-552f-4e42-ba7b-908daff1367f service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:30.393263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73011e-552f-4e42-ba7b-908daff1367f service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:30.393679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73011e-552f-4e42-ba7b-908daff1367f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:30.403857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73011e-552f-4e42-ba7b-908daff1367f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:30.405325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73011e-552f-4e42-ba7b-908daff1367f service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:30.415568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1001/2013] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:30.516456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef554a6c-a490-432e-b138-6b3f712de546 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:30.526838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef554a6c-a490-432e-b138-6b3f712de546 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:30.530614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989dea2-a3f4-4603-8766-c19a96f04655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:30.531261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989dea2-a3f4-4603-8766-c19a96f04655 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:30.531541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989dea2-a3f4-4603-8766-c19a96f04655 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:30.531814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989dea2-a3f4-4603-8766-c19a96f04655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:30.589304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989dea2-a3f4-4603-8766-c19a96f04655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:30.590824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989dea2-a3f4-4603-8766-c19a96f04655 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:30.600599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1013/2014] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:35.841444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a459b-5868-4c8f-9216-80a1fff48670 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:35.842290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a459b-5868-4c8f-9216-80a1fff48670 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:35.842511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a459b-5868-4c8f-9216-80a1fff48670 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:35.844238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1002/2015] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:54:35.853604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d98fb-909a-424c-8747-5942729b51e7 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:35.854423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d98fb-909a-424c-8747-5942729b51e7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:35.854423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d98fb-909a-424c-8747-5942729b51e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:35.875197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ab7d98fb-909a-424c-8747-5942729b51e7 None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:35.876486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ab7d98fb-909a-424c-8747-5942729b51e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:35.970182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1014/2016] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:54:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:35.985679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a459b-5868-4c8f-9216-80a1fff48670 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:35.987849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2f0a459b-5868-4c8f-9216-80a1fff48670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:36.007229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a459b-5868-4c8f-9216-80a1fff48670 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:36.013129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd99b45-e40b-4acc-9479-060466f4ac75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:36.013853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99b45-e40b-4acc-9479-060466f4ac75 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:36.014139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99b45-e40b-4acc-9479-060466f4ac75 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:36.014422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99b45-e40b-4acc-9479-060466f4ac75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:36.024377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd99b45-e40b-4acc-9479-060466f4ac75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:36.026072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd99b45-e40b-4acc-9479-060466f4ac75 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:36.037471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1003/2017] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:44.045221 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d98fb-909a-424c-8747-5942729b51e7 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:44.105684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:44.159068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d98fb-909a-424c-8747-5942729b51e7 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:44.164759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:44.165667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:44.166169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:44.166454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] PATH_INFO: `/v3/users/fafe7045c3774f38b4ddb9cdd52c4433` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:44.167937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafe7045c3774f38b4ddb9cdd52c4433)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:44.170508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:44.257599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafe7045c3774f38b4ddb9cdd52c4433'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.270516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafe7045c3774f38b4ddb9cdd52c4433'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.279614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafe7045c3774f38b4ddb9cdd52c4433'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.280219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafe7045c3774f38b4ddb9cdd52c4433'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.288844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafe7045c3774f38b4ddb9cdd52c4433'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.294793 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1015/2018] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:44 2022] DELETE /identity/v3/users/fafe7045c3774f38b4ddb9cdd52c4433 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:44.299453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1109c-74c2-49ba-abd4-b67c440c4e69 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:44.299908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1109c-74c2-49ba-abd4-b67c440c4e69 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:44.300040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1109c-74c2-49ba-abd4-b67c440c4e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:44.318879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-99f1109c-74c2-49ba-abd4-b67c440c4e69 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:44.320084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-99f1109c-74c2-49ba-abd4-b67c440c4e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:44.385498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1004/2019] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:44.396541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:44.398215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:44.464300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce89a02-6435-424a-9853-714374ddfaa0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:44.467963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef19c9-37c1-4b58-a971-f8de8282e119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:44.468626 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef19c9-37c1-4b58-a971-f8de8282e119 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:44.468909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef19c9-37c1-4b58-a971-f8de8282e119 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:44.469187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef19c9-37c1-4b58-a971-f8de8282e119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:44.478791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef19c9-37c1-4b58-a971-f8de8282e119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:44.480231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef19c9-37c1-4b58-a971-f8de8282e119 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:44.490367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1016/2020] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:44.770069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1109c-74c2-49ba-abd4-b67c440c4e69 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:44.771997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-99f1109c-74c2-49ba-abd4-b67c440c4e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:44.787806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1109c-74c2-49ba-abd4-b67c440c4e69 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:44.793926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:44.795145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:44.795430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:44.795710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] PATH_INFO: `/v3/projects/2716725ceb8e4e6a99c79930f7549d78` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:44.797009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2716725ceb8e4e6a99c79930f7549d78)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:44.799113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:44.855428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2716725ceb8e4e6a99c79930f7549d78', 'request_id': 'req-56400ade-bf30-4c49-888c-d2bc10539d8c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.875941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2716725ceb8e4e6a99c79930f7549d78', 'request_id': 'req-56400ade-bf30-4c49-888c-d2bc10539d8c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.894120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2716725ceb8e4e6a99c79930f7549d78', 'request_id': 'req-56400ade-bf30-4c49-888c-d2bc10539d8c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.913689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] The token cache is being invalidate because project 2716725ceb8e4e6a99c79930f7549d78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:44.914404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.915479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:44.921860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef19c9-37c1-4b58-a971-f8de8282e119 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:44.924297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1005/2021] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:44 2022] DELETE /identity/v3/projects/2716725ceb8e4e6a99c79930f7549d78 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:44.932234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef19c9-37c1-4b58-a971-f8de8282e119 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:45.011439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:45.022464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:45.024090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:45.024587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:45.025045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] PATH_INFO: `/v3/users/aed451e26abf49d8946d189789049f5a` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:45.027073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed451e26abf49d8946d189789049f5a)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:45.029236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:45.118245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed451e26abf49d8946d189789049f5a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.141900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed451e26abf49d8946d189789049f5a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.157798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed451e26abf49d8946d189789049f5a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.158705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed451e26abf49d8946d189789049f5a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.181312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d40c14fa-8272-4084-8054-f5d6a1ecdc36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed451e26abf49d8946d189789049f5a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.195501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1017/2022] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:54:44 2022] DELETE /identity/v3/users/aed451e26abf49d8946d189789049f5a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:45.634664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:45.636766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:45.660893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56400ade-bf30-4c49-888c-d2bc10539d8c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:45.723448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:45.744359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:45.745874 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:45.746334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:45.746777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] PATH_INFO: `/v3/projects/0e8ce980670840dcb79805a605be1861` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:45.748140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8ce980670840dcb79805a605be1861)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:45.750343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:45.798460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8ce980670840dcb79805a605be1861', 'request_id': 'req-7e18c694-c56a-44aa-b077-f6ade4f7a529'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.805097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8ce980670840dcb79805a605be1861', 'request_id': 'req-7e18c694-c56a-44aa-b077-f6ade4f7a529'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.813695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8ce980670840dcb79805a605be1861', 'request_id': 'req-7e18c694-c56a-44aa-b077-f6ade4f7a529'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.825385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] The token cache is being invalidate because project 0e8ce980670840dcb79805a605be1861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:54:45.825669 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.826368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7e18c694-c56a-44aa-b077-f6ade4f7a529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:54:45.827916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1006/2023] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:54:45 2022] DELETE /identity/v3/projects/0e8ce980670840dcb79805a605be1861 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:45.832924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42cf91-cef9-4708-8e17-c9eb86fdb5c9 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:45.833314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42cf91-cef9-4708-8e17-c9eb86fdb5c9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:45.833576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42cf91-cef9-4708-8e17-c9eb86fdb5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:45.853233 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6d42cf91-cef9-4708-8e17-c9eb86fdb5c9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:45.854497 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6d42cf91-cef9-4708-8e17-c9eb86fdb5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:45.913622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1018/2024] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:54:45.921871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48f74-f639-4132-8912-9928d5905417 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:45.922163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48f74-f639-4132-8912-9928d5905417 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:45.922439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48f74-f639-4132-8912-9928d5905417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:45.944312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-5fb48f74-f639-4132-8912-9928d5905417 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:45.945527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5fb48f74-f639-4132-8912-9928d5905417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:45.959026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1007/2025] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:45.964886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42cf91-cef9-4708-8e17-c9eb86fdb5c9 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:45.975415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42cf91-cef9-4708-8e17-c9eb86fdb5c9 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:45.981291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:45.982033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:45.982315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:45.982568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:45.983326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:54:45.983634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:45.985303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:45.998579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1019/2026] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:54:46.004716 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb48f74-f639-4132-8912-9928d5905417 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.014506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb48f74-f639-4132-8912-9928d5905417 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.020096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cb635-3ce1-46b2-9c37-c672e2e63c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.020893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cb635-3ce1-46b2-9c37-c672e2e63c8f admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.021215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cb635-3ce1-46b2-9c37-c672e2e63c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.021528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cb635-3ce1-46b2-9c37-c672e2e63c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.022344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cb635-3ce1-46b2-9c37-c672e2e63c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.024267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cb635-3ce1-46b2-9c37-c672e2e63c8f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.056433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1008/2027] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:46 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:46.057811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.068385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae867c-58c9-4c51-89e4-e66989ae7119 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.127198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0a83c-a7bc-459b-b28b-89b9c4e27fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.127887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a83c-a7bc-459b-b28b-89b9c4e27fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.128168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a83c-a7bc-459b-b28b-89b9c4e27fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.128445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0a83c-a7bc-459b-b28b-89b9c4e27fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.129170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0a83c-a7bc-459b-b28b-89b9c4e27fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.131236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0a83c-a7bc-459b-b28b-89b9c4e27fa3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.139269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:46.154385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1020/2028] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:46 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:54:46.159163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cb635-3ce1-46b2-9c37-c672e2e63c8f admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.170685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cb635-3ce1-46b2-9c37-c672e2e63c8f admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.176279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9de06-9128-4d17-8243-3f8a652bbc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.176962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9de06-9128-4d17-8243-3f8a652bbc71 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.177495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9de06-9128-4d17-8243-3f8a652bbc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.177749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9de06-9128-4d17-8243-3f8a652bbc71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.178645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9de06-9128-4d17-8243-3f8a652bbc71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.180254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9de06-9128-4d17-8243-3f8a652bbc71 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.191293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1009/2029] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:46.196038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0a83c-a7bc-459b-b28b-89b9c4e27fa3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.208280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0a83c-a7bc-459b-b28b-89b9c4e27fa3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.213590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2764ad6-c7cd-4156-b46d-14aa52af9161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.214320 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2764ad6-c7cd-4156-b46d-14aa52af9161 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.214601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2764ad6-c7cd-4156-b46d-14aa52af9161 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.214877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2764ad6-c7cd-4156-b46d-14aa52af9161 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.215618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2764ad6-c7cd-4156-b46d-14aa52af9161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.217234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2764ad6-c7cd-4156-b46d-14aa52af9161 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.228213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1021/2030] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:46.233032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9de06-9128-4d17-8243-3f8a652bbc71 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.247608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9de06-9128-4d17-8243-3f8a652bbc71 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.253000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b03e0-4fe9-4656-8323-8c6dab201d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.255384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b03e0-4fe9-4656-8323-8c6dab201d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.255818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b03e0-4fe9-4656-8323-8c6dab201d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.256171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b03e0-4fe9-4656-8323-8c6dab201d6b admin admin] PATH_INFO: `/v3/projects/325f8918f0f9436bb72569a891654da2/users/7202f0e9a8ee4092b7d51a53ed3be139/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.274500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b03e0-4fe9-4656-8323-8c6dab201d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=325f8918f0f9436bb72569a891654da2, user_id=7202f0e9a8ee4092b7d51a53ed3be139, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.278055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b03e0-4fe9-4656-8323-8c6dab201d6b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.306316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1010/2031] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:46 2022] PUT /identity/v3/projects/325f8918f0f9436bb72569a891654da2/users/7202f0e9a8ee4092b7d51a53ed3be139/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:54:46.316824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb44b9-9ac8-4bc7-972e-bb4a53b5fa97 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.317115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb44b9-9ac8-4bc7-972e-bb4a53b5fa97 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.317401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb44b9-9ac8-4bc7-972e-bb4a53b5fa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.359139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-0dfb44b9-9ac8-4bc7-972e-bb4a53b5fa97 None None] MFA Rules not processed for user `7202f0e9a8ee4092b7d51a53ed3be139`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:46.360696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0dfb44b9-9ac8-4bc7-972e-bb4a53b5fa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:46.435401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1022/2032] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:46.441107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b03e0-4fe9-4656-8323-8c6dab201d6b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.512026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b03e0-4fe9-4656-8323-8c6dab201d6b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.517151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b92d68-1eaa-43d7-bf50-ff2ef45b16ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.517913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b92d68-1eaa-43d7-bf50-ff2ef45b16ae admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.518209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b92d68-1eaa-43d7-bf50-ff2ef45b16ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.518512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b92d68-1eaa-43d7-bf50-ff2ef45b16ae admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.519205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b92d68-1eaa-43d7-bf50-ff2ef45b16ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.521189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b92d68-1eaa-43d7-bf50-ff2ef45b16ae admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.536346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:46.553944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1011/2033] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:54:46 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:46.558151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb44b9-9ac8-4bc7-972e-bb4a53b5fa97 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.570306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb44b9-9ac8-4bc7-972e-bb4a53b5fa97 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.633206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db9c5f-3bbd-49e5-a0da-7ae4574dad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.633957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9c5f-3bbd-49e5-a0da-7ae4574dad63 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.634247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9c5f-3bbd-49e5-a0da-7ae4574dad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.634526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db9c5f-3bbd-49e5-a0da-7ae4574dad63 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.635244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db9c5f-3bbd-49e5-a0da-7ae4574dad63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.637184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db9c5f-3bbd-49e5-a0da-7ae4574dad63 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.667656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1023/2034] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:54:46 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:54:46.668953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b92d68-1eaa-43d7-bf50-ff2ef45b16ae admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.687100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b92d68-1eaa-43d7-bf50-ff2ef45b16ae admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.694843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a417335-2807-4719-89ad-dc5a81a74739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.696157 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a417335-2807-4719-89ad-dc5a81a74739 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.696565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a417335-2807-4719-89ad-dc5a81a74739 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.696987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a417335-2807-4719-89ad-dc5a81a74739 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.697743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a417335-2807-4719-89ad-dc5a81a74739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.700349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a417335-2807-4719-89ad-dc5a81a74739 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.712972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1012/2035] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:46.717756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db9c5f-3bbd-49e5-a0da-7ae4574dad63 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.729903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db9c5f-3bbd-49e5-a0da-7ae4574dad63 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.737994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920c965-96c6-4182-884d-08b6393cb054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.738199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920c965-96c6-4182-884d-08b6393cb054 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.738613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920c965-96c6-4182-884d-08b6393cb054 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.739095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920c965-96c6-4182-884d-08b6393cb054 admin admin] PATH_INFO: `/v3/projects/f341abba58b74b9facf5e1aa42d0f933/users/a3c5275a337849a7ae9b849492e8ec06/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.764554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920c965-96c6-4182-884d-08b6393cb054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f341abba58b74b9facf5e1aa42d0f933, user_id=a3c5275a337849a7ae9b849492e8ec06, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.767333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920c965-96c6-4182-884d-08b6393cb054 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.782104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1024/2036] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:46 2022] PUT /identity/v3/projects/f341abba58b74b9facf5e1aa42d0f933/users/a3c5275a337849a7ae9b849492e8ec06/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:46.786052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a417335-2807-4719-89ad-dc5a81a74739 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.796186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a417335-2807-4719-89ad-dc5a81a74739 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.858016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b3b60-7623-4102-afbb-f340a55a2d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.858692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b3b60-7623-4102-afbb-f340a55a2d8c admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.858970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b3b60-7623-4102-afbb-f340a55a2d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.859241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b3b60-7623-4102-afbb-f340a55a2d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.859946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b3b60-7623-4102-afbb-f340a55a2d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.861531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b3b60-7623-4102-afbb-f340a55a2d8c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.872438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1013/2037] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:54:46.876474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920c965-96c6-4182-884d-08b6393cb054 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:46.887834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920c965-96c6-4182-884d-08b6393cb054 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:46.892992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:46.893930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.894191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.894501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] PATH_INFO: `/v3/projects/f341abba58b74b9facf5e1aa42d0f933/users/a3c5275a337849a7ae9b849492e8ec06/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.897290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f341abba58b74b9facf5e1aa42d0f933, user_id=a3c5275a337849a7ae9b849492e8ec06, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:46.900023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:46.912250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1025/2038] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:54:46 2022] PUT /identity/v3/projects/f341abba58b74b9facf5e1aa42d0f933/users/a3c5275a337849a7ae9b849492e8ec06/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:54:46.923618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd837520-f43a-4f27-8cdd-ef558d558d6a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:46.923911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd837520-f43a-4f27-8cdd-ef558d558d6a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:46.924164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd837520-f43a-4f27-8cdd-ef558d558d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:46.953753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-fd837520-f43a-4f27-8cdd-ef558d558d6a None None] MFA Rules not processed for user `a3c5275a337849a7ae9b849492e8ec06`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:54:46.955005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fd837520-f43a-4f27-8cdd-ef558d558d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:47.024105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1014/2039] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:54:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:54:47.035378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:54:47.037743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:54:47.067065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:47.111785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f8ae3-ecef-462f-be20-b9693cd461a8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:54:47.117240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f70714-5b6f-4840-a593-31565db47786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:54:47.118246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70714-5b6f-4840-a593-31565db47786 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:54:47.118623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70714-5b6f-4840-a593-31565db47786 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:54:47.118980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f70714-5b6f-4840-a593-31565db47786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:54:47.181446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f70714-5b6f-4840-a593-31565db47786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:54:47.182938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f70714-5b6f-4840-a593-31565db47786 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:54:47.191866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1026/2040] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:54:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:54:57.020772 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=70 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:54:57.187909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=38 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:03.787436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd837520-f43a-4f27-8cdd-ef558d558d6a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:03.789048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fd837520-f43a-4f27-8cdd-ef558d558d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:03.879745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd837520-f43a-4f27-8cdd-ef558d558d6a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:03.886019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:03.886931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:03.887207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:03.887485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] PATH_INFO: `/v3/users/04c4bc187e474b5da94ef65b48c0778a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:03.888782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c4bc187e474b5da94ef65b48c0778a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:03.891447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:03.966331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c4bc187e474b5da94ef65b48c0778a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:03.998627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c4bc187e474b5da94ef65b48c0778a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:04.009078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c4bc187e474b5da94ef65b48c0778a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:04.019862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c4bc187e474b5da94ef65b48c0778a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:04.021528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c4bc187e474b5da94ef65b48c0778a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:04.031870 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1015/2041] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:03 2022] DELETE /identity/v3/users/04c4bc187e474b5da94ef65b48c0778a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:04.033938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983eec42-e839-4372-93f6-641399bfb53c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:04.034409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983eec42-e839-4372-93f6-641399bfb53c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:04.034703 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983eec42-e839-4372-93f6-641399bfb53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:04.056021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-983eec42-e839-4372-93f6-641399bfb53c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:04.058104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-983eec42-e839-4372-93f6-641399bfb53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:04.236610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1027/2042] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:04.290491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:04.293676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:04.390723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983eec42-e839-4372-93f6-641399bfb53c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:04.393045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-983eec42-e839-4372-93f6-641399bfb53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:04.414757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d1e49-6f56-4552-b694-e9090bbce00e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:04.418689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:04.419497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:04.419798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:04.420234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:04.436443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:04.439031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:04.461621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1016/2043] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:04.474613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983eec42-e839-4372-93f6-641399bfb53c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:04.480044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a1065-2312-417d-ba9b-bca80c4e84c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:04.480754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1065-2312-417d-ba9b-bca80c4e84c0 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:04.481218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1065-2312-417d-ba9b-bca80c4e84c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:04.481595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a1065-2312-417d-ba9b-bca80c4e84c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:04.484632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-886a1065-2312-417d-ba9b-bca80c4e84c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:04.564689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a1065-2312-417d-ba9b-bca80c4e84c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:04.566442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a1065-2312-417d-ba9b-bca80c4e84c0 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:04.580438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1028/2044] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:04.800221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:04.801665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:04.831138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9e8b1-9b9f-4d88-b20e-21ee604acdc3 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:04.845500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:04.845500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:04.845500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:04.845500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] PATH_INFO: `/v3/users/6a9ba88b730f481796109d38ba786e05` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:04.849371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9ba88b730f481796109d38ba786e05)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:04.854637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:04.893740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:04.951707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9ba88b730f481796109d38ba786e05'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:04.980601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9ba88b730f481796109d38ba786e05'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:04.999054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9ba88b730f481796109d38ba786e05'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:05.009742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9ba88b730f481796109d38ba786e05'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:05.011558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9ba88b730f481796109d38ba786e05'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:05.028544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1017/2045] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:04 2022] DELETE /identity/v3/users/6a9ba88b730f481796109d38ba786e05 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:05.041011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759983d-5a7a-4805-b581-cc2bf6dba391 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:05.042614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759983d-5a7a-4805-b581-cc2bf6dba391 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:05.043156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759983d-5a7a-4805-b581-cc2bf6dba391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:05.072530 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d759983d-5a7a-4805-b581-cc2bf6dba391 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:05.074407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d759983d-5a7a-4805-b581-cc2bf6dba391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:05.196300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:05.225593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1029/2046] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:05.240810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:05.250599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:05.320584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655d310-b231-46e3-9c46-2ac3d95def96 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:05.325212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd228a-76dc-42e3-8fd4-d67875d035d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:05.326071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd228a-76dc-42e3-8fd4-d67875d035d2 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:05.326354 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd228a-76dc-42e3-8fd4-d67875d035d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:05.326629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd228a-76dc-42e3-8fd4-d67875d035d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:05.338167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd228a-76dc-42e3-8fd4-d67875d035d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:05.339714 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd228a-76dc-42e3-8fd4-d67875d035d2 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:05.355019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1018/2047] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:05.567526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759983d-5a7a-4805-b581-cc2bf6dba391 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:05.571007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d759983d-5a7a-4805-b581-cc2bf6dba391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:05.591947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759983d-5a7a-4805-b581-cc2bf6dba391 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:05.600091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:05.601314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:05.602454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:05.606614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] PATH_INFO: `/v3/projects/6621af629a454b97b6879705994e43ea` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:05.608033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6621af629a454b97b6879705994e43ea)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:05.610096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:05.704561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6621af629a454b97b6879705994e43ea', 'request_id': 'req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:05.721753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6621af629a454b97b6879705994e43ea', 'request_id': 'req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:05.742105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6621af629a454b97b6879705994e43ea', 'request_id': 'req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:05.771423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] The token cache is being invalidate because project 6621af629a454b97b6879705994e43ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:05.772124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:05.773053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:05.775505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1030/2048] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:05 2022] DELETE /identity/v3/projects/6621af629a454b97b6879705994e43ea => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:05.779492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd228a-76dc-42e3-8fd4-d67875d035d2 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:05.811430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd228a-76dc-42e3-8fd4-d67875d035d2 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:05.945554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:05.946630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:05.947074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:05.947633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] PATH_INFO: `/v3/users/531a0e8ad83b4b5e83b5e5b12b230d04` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:05.951778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=531a0e8ad83b4b5e83b5e5b12b230d04)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:05.956171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:06.046690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:06.056424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '531a0e8ad83b4b5e83b5e5b12b230d04'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.092942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '531a0e8ad83b4b5e83b5e5b12b230d04'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.113730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '531a0e8ad83b4b5e83b5e5b12b230d04'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.129357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '531a0e8ad83b4b5e83b5e5b12b230d04'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.131063 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-6a69f063-45aa-4c75-8507-74d6286af11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '531a0e8ad83b4b5e83b5e5b12b230d04'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.157062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1019/2049] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:05 2022] DELETE /identity/v3/users/531a0e8ad83b4b5e83b5e5b12b230d04 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:06.336374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:06.338150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:06.366480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380d35a-5e59-4ab2-a920-b76c2ff1a7c7 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:06.503806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:06.506446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:06.508000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:06.508558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] PATH_INFO: `/v3/projects/ddcdf331c2974c37a4c248bbc04912f7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:06.511325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddcdf331c2974c37a4c248bbc04912f7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:06.513905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:06.627598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddcdf331c2974c37a4c248bbc04912f7', 'request_id': 'req-eac82cd6-c831-4b12-aad3-f954235adf3a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.642091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddcdf331c2974c37a4c248bbc04912f7', 'request_id': 'req-eac82cd6-c831-4b12-aad3-f954235adf3a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.666576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddcdf331c2974c37a4c248bbc04912f7', 'request_id': 'req-eac82cd6-c831-4b12-aad3-f954235adf3a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.698905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] The token cache is being invalidate because project ddcdf331c2974c37a4c248bbc04912f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:06.699211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.700330 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eac82cd6-c831-4b12-aad3-f954235adf3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:06.710669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1031/2050] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:06 2022] DELETE /identity/v3/projects/ddcdf331c2974c37a4c248bbc04912f7 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:06.714625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680488-38e0-44dd-880a-89df3d5d1ca1 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:06.714919 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680488-38e0-44dd-880a-89df3d5d1ca1 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:06.715181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680488-38e0-44dd-880a-89df3d5d1ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:06.812412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-06680488-38e0-44dd-880a-89df3d5d1ca1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:06.816501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-06680488-38e0-44dd-880a-89df3d5d1ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:06.982391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1020/2051] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:06.993695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090962-a29d-491e-b229-6afeb53aeb25 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.000960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090962-a29d-491e-b229-6afeb53aeb25 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.001402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090962-a29d-491e-b229-6afeb53aeb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.023557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:07.057359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-82090962-a29d-491e-b229-6afeb53aeb25 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:07.065258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-82090962-a29d-491e-b229-6afeb53aeb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:07.083328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1032/2052] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:07.089148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06680488-38e0-44dd-880a-89df3d5d1ca1 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.105566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82090962-a29d-491e-b229-6afeb53aeb25 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.112724 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-82090962-a29d-491e-b229-6afeb53aeb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:07.113724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06680488-38e0-44dd-880a-89df3d5d1ca1 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.124962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.128883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.129752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.130763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.133195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:55:07.134195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.138548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.148171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82090962-a29d-491e-b229-6afeb53aeb25 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.158514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1021/2053] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:55:07.159412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.160391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.160723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.161023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] PATH_INFO: `/v3/projects/b964173ff6b34e7eb9153970dd2b5b6c` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.163891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.165732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b964173ff6b34e7eb9153970dd2b5b6c)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.169656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.192406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34ce2f-c429-4d1a-b2ae-6a16d41d10fb admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.208141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688f033-cad5-4aaa-aeca-d2794d3db571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.209099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688f033-cad5-4aaa-aeca-d2794d3db571 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.209479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688f033-cad5-4aaa-aeca-d2794d3db571 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.209866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688f033-cad5-4aaa-aeca-d2794d3db571 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.213327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688f033-cad5-4aaa-aeca-d2794d3db571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.215482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688f033-cad5-4aaa-aeca-d2794d3db571 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.293064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1022/2054] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:55:07 2022] POST /identity/v3/projects => generated 361 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:07.295242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688f033-cad5-4aaa-aeca-d2794d3db571 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.301034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b964173ff6b34e7eb9153970dd2b5b6c', 'request_id': 'req-6abe3558-e22d-453f-a806-82876f258b3c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:07.316809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b964173ff6b34e7eb9153970dd2b5b6c', 'request_id': 'req-6abe3558-e22d-453f-a806-82876f258b3c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:07.329672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b964173ff6b34e7eb9153970dd2b5b6c', 'request_id': 'req-6abe3558-e22d-453f-a806-82876f258b3c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:07.347612 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] The token cache is being invalidate because project b964173ff6b34e7eb9153970dd2b5b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:07.349340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:07.350500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6abe3558-e22d-453f-a806-82876f258b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:07.357064 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1033/2055] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:07 2022] DELETE /identity/v3/projects/b964173ff6b34e7eb9153970dd2b5b6c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:07.358841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83cf1-ad03-4c8f-8efa-58decee831cb None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.360039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83cf1-ad03-4c8f-8efa-58decee831cb None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.360632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83cf1-ad03-4c8f-8efa-58decee831cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.372008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688f033-cad5-4aaa-aeca-d2794d3db571 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.377686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.378691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.379256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.381533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.382497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.385297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.419636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1023/2056] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:55:07 2022] POST /identity/v3/users => generated 281 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:07.423102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.425800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-10b83cf1-ad03-4c8f-8efa-58decee831cb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:07.429119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-10b83cf1-ad03-4c8f-8efa-58decee831cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:07.430666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:07.448610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1034/2057] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:07.458069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb024d25-730f-4bbe-8f68-3a7a93d29995 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.459055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5345b97-8575-49af-8686-afc71b502870 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.462938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5345b97-8575-49af-8686-afc71b502870 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.463488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5345b97-8575-49af-8686-afc71b502870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.472739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6619-92b8-43c3-89b0-31a6b91605bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.473657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6619-92b8-43c3-89b0-31a6b91605bd admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.474030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6619-92b8-43c3-89b0-31a6b91605bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.474397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6619-92b8-43c3-89b0-31a6b91605bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.475152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa6619-92b8-43c3-89b0-31a6b91605bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.481440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa6619-92b8-43c3-89b0-31a6b91605bd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.493875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:07.511751 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a5345b97-8575-49af-8686-afc71b502870 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:07.511995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1024/2058] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:07.513493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a5345b97-8575-49af-8686-afc71b502870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:07.515743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6619-92b8-43c3-89b0-31a6b91605bd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.527243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1035/2059] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:07.533019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5345b97-8575-49af-8686-afc71b502870 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.536764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6619-92b8-43c3-89b0-31a6b91605bd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.547609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de561c-0a65-4bf6-af56-ba2406d2ffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.548838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5345b97-8575-49af-8686-afc71b502870 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.549579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de561c-0a65-4bf6-af56-ba2406d2ffb0 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.549976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de561c-0a65-4bf6-af56-ba2406d2ffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.550288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de561c-0a65-4bf6-af56-ba2406d2ffb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.552250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de561c-0a65-4bf6-af56-ba2406d2ffb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.556350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de561c-0a65-4bf6-af56-ba2406d2ffb0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.581884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.581884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.581884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.581884 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.582342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:55:07.582342 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.582502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1025/2060] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:07.584410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de561c-0a65-4bf6-af56-ba2406d2ffb0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.588617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.606252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de561c-0a65-4bf6-af56-ba2406d2ffb0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.621564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4dfe6-54d7-4419-8130-6ca0b68f7893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.628881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4dfe6-54d7-4419-8130-6ca0b68f7893 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.628881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4dfe6-54d7-4419-8130-6ca0b68f7893 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.628881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4dfe6-54d7-4419-8130-6ca0b68f7893 admin admin] PATH_INFO: `/v3/projects/0e13d8605b3249bcb63c5e06cba1bb0b/users/fda36db6863b4470b7e5da3e3d9f086c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.629338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1036/2061] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:55:07.634804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.659909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4dfe6-54d7-4419-8130-6ca0b68f7893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e13d8605b3249bcb63c5e06cba1bb0b, user_id=fda36db6863b4470b7e5da3e3d9f086c, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.664831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be1cb5-9698-4771-82bb-357f68beb18d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.666784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4dfe6-54d7-4419-8130-6ca0b68f7893 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.681474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5c744-fa33-48f4-b005-c97837e1aeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.682461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5c744-fa33-48f4-b005-c97837e1aeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.682822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5c744-fa33-48f4-b005-c97837e1aeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.683549 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5c744-fa33-48f4-b005-c97837e1aeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.684433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5c744-fa33-48f4-b005-c97837e1aeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.688256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5c744-fa33-48f4-b005-c97837e1aeb2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.709065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1026/2062] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:07 2022] PUT /identity/v3/projects/0e13d8605b3249bcb63c5e06cba1bb0b/users/fda36db6863b4470b7e5da3e3d9f086c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:07.711972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0be77e-3790-4af4-9624-c35e417dc850 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.713590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0be77e-3790-4af4-9624-c35e417dc850 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.713940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0be77e-3790-4af4-9624-c35e417dc850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.745157 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1037/2063] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:55:07 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:07.748086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5c744-fa33-48f4-b005-c97837e1aeb2 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.768240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ae0be77e-3790-4af4-9624-c35e417dc850 None None] MFA Rules not processed for user `fda36db6863b4470b7e5da3e3d9f086c`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:07.773922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ae0be77e-3790-4af4-9624-c35e417dc850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:07.840111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5c744-fa33-48f4-b005-c97837e1aeb2 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.847969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7f57-2738-42c1-abcb-f87f65b20615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.849633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7f57-2738-42c1-abcb-f87f65b20615 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.850103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7f57-2738-42c1-abcb-f87f65b20615 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.850753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7f57-2738-42c1-abcb-f87f65b20615 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.852998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae7f57-2738-42c1-abcb-f87f65b20615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.856610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae7f57-2738-42c1-abcb-f87f65b20615 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.894218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1038/2064] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:55:07 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:07.898424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7f57-2738-42c1-abcb-f87f65b20615 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.917440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7f57-2738-42c1-abcb-f87f65b20615 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.923715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1027/2065] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:07.925120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.927422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.927959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.929093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.929682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.931565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:07.944517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1039/2066] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:07.948769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0be77e-3790-4af4-9624-c35e417dc850 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.967586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0be77e-3790-4af4-9624-c35e417dc850 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:07.970022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:07.972597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:07.980883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f24372-fe39-46ac-aa02-8ea2c5a0ff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:07.981715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f24372-fe39-46ac-aa02-8ea2c5a0ff0a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:07.982100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f24372-fe39-46ac-aa02-8ea2c5a0ff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:07.982468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f24372-fe39-46ac-aa02-8ea2c5a0ff0a admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:07.983514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f24372-fe39-46ac-aa02-8ea2c5a0ff0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:07.985168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f24372-fe39-46ac-aa02-8ea2c5a0ff0a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:08.002072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:08.019693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1028/2067] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:08.021200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f24372-fe39-46ac-aa02-8ea2c5a0ff0a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:08.041915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f24372-fe39-46ac-aa02-8ea2c5a0ff0a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:08.058008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b7e56-dfb4-4963-b717-98cc7c466d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:08.058008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b7e56-dfb4-4963-b717-98cc7c466d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:08.058462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b7e56-dfb4-4963-b717-98cc7c466d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:08.058784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b7e56-dfb4-4963-b717-98cc7c466d8f admin admin] PATH_INFO: `/v3/projects/b2b9cfafc0fa4867a7b842745102a5a8/users/9e6f576445a141cbb63ad3882e258ecb/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:08.101243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b7e56-dfb4-4963-b717-98cc7c466d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b9cfafc0fa4867a7b842745102a5a8, user_id=9e6f576445a141cbb63ad3882e258ecb, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:08.104468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b7e56-dfb4-4963-b717-98cc7c466d8f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:08.140284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1029/2068] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:08 2022] PUT /identity/v3/projects/b2b9cfafc0fa4867a7b842745102a5a8/users/9e6f576445a141cbb63ad3882e258ecb/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:08.142208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4d5e-14dd-4246-86a8-cb108845074f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:08.142517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4d5e-14dd-4246-86a8-cb108845074f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:08.142820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4d5e-14dd-4246-86a8-cb108845074f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:08.165228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc719e6-0e18-4292-a993-17451706f6cc admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:08.205750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-15dd4d5e-14dd-4246-86a8-cb108845074f None None] MFA Rules not processed for user `9e6f576445a141cbb63ad3882e258ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:08.209167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-15dd4d5e-14dd-4246-86a8-cb108845074f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:08.340883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b68e0d-155e-445e-8495-e64286a43dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:08.342778 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68e0d-155e-445e-8495-e64286a43dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:08.345263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68e0d-155e-445e-8495-e64286a43dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:08.345628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b68e0d-155e-445e-8495-e64286a43dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:08.347559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1030/2069] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:55:08.352050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4d5e-14dd-4246-86a8-cb108845074f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:08.382464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:08.523929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b68e0d-155e-445e-8495-e64286a43dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:08.525525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b68e0d-155e-445e-8495-e64286a43dc7 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:08.536668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4d5e-14dd-4246-86a8-cb108845074f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:08.551529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717f681-9337-411d-beca-e61446676627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:08.552357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717f681-9337-411d-beca-e61446676627 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:08.552779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717f681-9337-411d-beca-e61446676627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:08.553055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717f681-9337-411d-beca-e61446676627 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:08.553794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717f681-9337-411d-beca-e61446676627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:08.554514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1040/2070] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:08.560102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717f681-9337-411d-beca-e61446676627 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:08.571655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b68e0d-155e-445e-8495-e64286a43dc7 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:08.590581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b68e0d-155e-445e-8495-e64286a43dc7 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:08.602166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:08.603575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:08.603575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:08.603980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] PATH_INFO: `/v3/users/fda36db6863b4470b7e5da3e3d9f086c` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:08.606988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda36db6863b4470b7e5da3e3d9f086c)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:08.611292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:08.640695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1031/2071] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:55:08 2022] POST /identity/v3/projects => generated 389 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:55:08.644030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717f681-9337-411d-beca-e61446676627 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:08.742090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda36db6863b4470b7e5da3e3d9f086c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:08.769426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda36db6863b4470b7e5da3e3d9f086c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:08.797155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda36db6863b4470b7e5da3e3d9f086c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:08.798744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda36db6863b4470b7e5da3e3d9f086c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:08.810473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:08.815425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e473eb47-4056-4a22-beaa-435c6e9981fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda36db6863b4470b7e5da3e3d9f086c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:08.842499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1041/2072] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:08 2022] DELETE /identity/v3/users/fda36db6863b4470b7e5da3e3d9f086c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:55:08.843957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9978-e940-4733-a0ec-91c738a1b8e5 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:08.844303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9978-e940-4733-a0ec-91c738a1b8e5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:08.844528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9978-e940-4733-a0ec-91c738a1b8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:08.880408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717f681-9337-411d-beca-e61446676627 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:08.915727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c01d9978-e940-4733-a0ec-91c738a1b8e5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:08.916956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c01d9978-e940-4733-a0ec-91c738a1b8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:09.035761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:09.036620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:09.036917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:09.037176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:09.037938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:09.054297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:09.060498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1042/2073] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:09.095158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d9978-e940-4733-a0ec-91c738a1b8e5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:09.096877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c01d9978-e940-4733-a0ec-91c738a1b8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:09.126367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1032/2074] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:55:08 2022] POST /identity/v3/users => generated 301 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:09.127367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:09.131445 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:09.153025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7e9a4-f23a-4b60-b32b-25b377b5034a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:09.168202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25d11f-0393-414c-98bd-af26b32b69b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:09.170116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25d11f-0393-414c-98bd-af26b32b69b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:09.170985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25d11f-0393-414c-98bd-af26b32b69b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:09.171478 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25d11f-0393-414c-98bd-af26b32b69b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:09.172588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25d11f-0393-414c-98bd-af26b32b69b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:09.174517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db25d11f-0393-414c-98bd-af26b32b69b7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:09.196956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1033/2075] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:09.204924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25d11f-0393-414c-98bd-af26b32b69b7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:09.214236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d9978-e940-4733-a0ec-91c738a1b8e5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:09.225805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b02746-69ae-473b-8e60-1e77d893a0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:09.226725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02746-69ae-473b-8e60-1e77d893a0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:09.227428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02746-69ae-473b-8e60-1e77d893a0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:09.228629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b02746-69ae-473b-8e60-1e77d893a0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:09.230217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25d11f-0393-414c-98bd-af26b32b69b7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:09.246339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5e891-ad51-4423-bb5a-67b36035957e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:09.247332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5e891-ad51-4423-bb5a-67b36035957e admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:09.247718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5e891-ad51-4423-bb5a-67b36035957e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:09.248165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5e891-ad51-4423-bb5a-67b36035957e admin admin] PATH_INFO: `/v3/projects/06c96fcf71694e94b31403913afd9446/users/2c1e902ee2d0420f8709d93432c47ef7/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:09.250273 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b02746-69ae-473b-8e60-1e77d893a0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:09.257731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b02746-69ae-473b-8e60-1e77d893a0b3 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:09.281185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1043/2076] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:09.287061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5e891-ad51-4423-bb5a-67b36035957e admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c96fcf71694e94b31403913afd9446, user_id=2c1e902ee2d0420f8709d93432c47ef7, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:09.290954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5e891-ad51-4423-bb5a-67b36035957e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:09.296329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:09.314439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1034/2077] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:09 2022] PUT /identity/v3/projects/06c96fcf71694e94b31403913afd9446/users/2c1e902ee2d0420f8709d93432c47ef7/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:09.318648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b02746-69ae-473b-8e60-1e77d893a0b3 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:09.339008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b02746-69ae-473b-8e60-1e77d893a0b3 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:09.505287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05976c5d-6438-4c88-818d-65eacdc17648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:09.506197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05976c5d-6438-4c88-818d-65eacdc17648 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:09.506654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05976c5d-6438-4c88-818d-65eacdc17648 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:09.507090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05976c5d-6438-4c88-818d-65eacdc17648 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:09.508078 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05976c5d-6438-4c88-818d-65eacdc17648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:09.510086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05976c5d-6438-4c88-818d-65eacdc17648 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:09.526033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1044/2078] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:09.530386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5e891-ad51-4423-bb5a-67b36035957e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:09.550035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5e891-ad51-4423-bb5a-67b36035957e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:09.557204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:09.558382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:09.558723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:09.559060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] PATH_INFO: `/v3/projects/06c96fcf71694e94b31403913afd9446/users/2c1e902ee2d0420f8709d93432c47ef7/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:09.562154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c96fcf71694e94b31403913afd9446, user_id=2c1e902ee2d0420f8709d93432c47ef7, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:09.565954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:09.594142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1035/2079] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:09 2022] PUT /identity/v3/projects/06c96fcf71694e94b31403913afd9446/users/2c1e902ee2d0420f8709d93432c47ef7/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:09.597058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9202a6-8034-4654-9b9c-16c03986407f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:09.597499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9202a6-8034-4654-9b9c-16c03986407f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:09.598148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9202a6-8034-4654-9b9c-16c03986407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:09.651776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1e9202a6-8034-4654-9b9c-16c03986407f None None] MFA Rules not processed for user `2c1e902ee2d0420f8709d93432c47ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:09.654519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1e9202a6-8034-4654-9b9c-16c03986407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:09.693842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:09.770385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1045/2080] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:09.779814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:09.782023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:09.878676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a076e6c-cfa5-4862-995a-bff110e72af4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:09.883945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:09.884715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:09.884715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:09.885082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:09.896232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:09.898110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:09.910959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1036/2081] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:10.108970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9202a6-8034-4654-9b9c-16c03986407f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:10.205633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:10.206655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:10.213274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9202a6-8034-4654-9b9c-16c03986407f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:10.221468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:10.223495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.223986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.224399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] PATH_INFO: `/v3/users/9e6f576445a141cbb63ad3882e258ecb` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.225941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e6f576445a141cbb63ad3882e258ecb)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:10.228165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:10.252958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876102b-3625-4d11-9bf6-8dc117a50da4 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:10.272148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:10.273853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.274366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.274886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] PATH_INFO: `/v3/projects/0e13d8605b3249bcb63c5e06cba1bb0b` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.276682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e13d8605b3249bcb63c5e06cba1bb0b)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:10.279198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:10.319867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e6f576445a141cbb63ad3882e258ecb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.337406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e6f576445a141cbb63ad3882e258ecb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.349063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e6f576445a141cbb63ad3882e258ecb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.350252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e6f576445a141cbb63ad3882e258ecb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.360346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e13d8605b3249bcb63c5e06cba1bb0b', 'request_id': 'req-103d1739-90f1-4f8e-9618-4dd855ea0f99'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.362070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e950425c-084b-4892-9b97-531b75712a78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e6f576445a141cbb63ad3882e258ecb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.369514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e13d8605b3249bcb63c5e06cba1bb0b', 'request_id': 'req-103d1739-90f1-4f8e-9618-4dd855ea0f99'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.370387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1046/2082] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:10 2022] DELETE /identity/v3/users/9e6f576445a141cbb63ad3882e258ecb => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:10.381447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cfd5-1654-4cca-8e2d-62ebb6ef0976 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.381730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cfd5-1654-4cca-8e2d-62ebb6ef0976 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.382050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cfd5-1654-4cca-8e2d-62ebb6ef0976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.383427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e13d8605b3249bcb63c5e06cba1bb0b', 'request_id': 'req-103d1739-90f1-4f8e-9618-4dd855ea0f99'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.396037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] The token cache is being invalidate because project 0e13d8605b3249bcb63c5e06cba1bb0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:10.396339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.397447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-103d1739-90f1-4f8e-9618-4dd855ea0f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.402136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1037/2083] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:10 2022] DELETE /identity/v3/projects/0e13d8605b3249bcb63c5e06cba1bb0b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:10.404561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2aa5-7d6a-4e44-8bd5-b3e8e1962f49 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.404853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2aa5-7d6a-4e44-8bd5-b3e8e1962f49 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.406179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b2aa5-7d6a-4e44-8bd5-b3e8e1962f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.417400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6021cfd5-1654-4cca-8e2d-62ebb6ef0976 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:10.418150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6021cfd5-1654-4cca-8e2d-62ebb6ef0976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:10.430970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8f0b2aa5-7d6a-4e44-8bd5-b3e8e1962f49 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:10.432539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8f0b2aa5-7d6a-4e44-8bd5-b3e8e1962f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:10.465110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:10.530243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1047/2084] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:10.531506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021cfd5-1654-4cca-8e2d-62ebb6ef0976 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:10.532991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6021cfd5-1654-4cca-8e2d-62ebb6ef0976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:10.558800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021cfd5-1654-4cca-8e2d-62ebb6ef0976 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:10.560671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1038/2085] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:10.562404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b2aa5-7d6a-4e44-8bd5-b3e8e1962f49 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:10.564457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8f0b2aa5-7d6a-4e44-8bd5-b3e8e1962f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:10.566354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:10.567327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.567605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.567983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] PATH_INFO: `/v3/users/7202f0e9a8ee4092b7d51a53ed3be139` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.569499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7202f0e9a8ee4092b7d51a53ed3be139)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:10.571559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:10.599329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:10.656575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7202f0e9a8ee4092b7d51a53ed3be139'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.685668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7202f0e9a8ee4092b7d51a53ed3be139'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.697038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b2aa5-7d6a-4e44-8bd5-b3e8e1962f49 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:10.700416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7202f0e9a8ee4092b7d51a53ed3be139'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.701168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7202f0e9a8ee4092b7d51a53ed3be139'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.705371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168c040-656c-4af2-aceb-f803f7f40e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:10.706389 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c040-656c-4af2-aceb-f803f7f40e22 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.706831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c040-656c-4af2-aceb-f803f7f40e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.707285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168c040-656c-4af2-aceb-f803f7f40e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.710813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e84d956f-3b57-457e-be90-6790ff88f086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7202f0e9a8ee4092b7d51a53ed3be139'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:10.717912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1048/2086] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:10 2022] DELETE /identity/v3/users/7202f0e9a8ee4092b7d51a53ed3be139 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:10.719359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0268-581e-4045-80f6-8e6047c32657 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.719678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0268-581e-4045-80f6-8e6047c32657 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.719958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf0268-581e-4045-80f6-8e6047c32657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.727340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168c040-656c-4af2-aceb-f803f7f40e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:10.729212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168c040-656c-4af2-aceb-f803f7f40e22 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:10.742780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a7bf0268-581e-4045-80f6-8e6047c32657 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:10.744114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a7bf0268-581e-4045-80f6-8e6047c32657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:10.747070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1039/2087] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:10.748646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed631b2-8c88-49ce-ac4c-298fcaa8d5db None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.749052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed631b2-8c88-49ce-ac4c-298fcaa8d5db None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.749420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed631b2-8c88-49ce-ac4c-298fcaa8d5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.770358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-eed631b2-8c88-49ce-ac4c-298fcaa8d5db None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:10.771580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-eed631b2-8c88-49ce-ac4c-298fcaa8d5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:10.867895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1049/2088] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:10.872100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf0268-581e-4045-80f6-8e6047c32657 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:10.883440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1040/2089] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:10.889662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf0268-581e-4045-80f6-8e6047c32657 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:10.901303 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:10.902142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:10.902706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:10.905274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:10.906132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:55:10.906452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:10.908265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:10.910300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed631b2-8c88-49ce-ac4c-298fcaa8d5db None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:10.912209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-eed631b2-8c88-49ce-ac4c-298fcaa8d5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:10.960570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1050/2090] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:55:10.967133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.024990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad39c-2dbe-4296-a2c2-7cee424895c4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.032032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed631b2-8c88-49ce-ac4c-298fcaa8d5db None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.036648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1ca8e-7f79-4db9-95fa-7dbebaec265d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.037382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ca8e-7f79-4db9-95fa-7dbebaec265d service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.037719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ca8e-7f79-4db9-95fa-7dbebaec265d service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.038527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1ca8e-7f79-4db9-95fa-7dbebaec265d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.041251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.042371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.043970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.044328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.045194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.049922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1ca8e-7f79-4db9-95fa-7dbebaec265d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.050565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.051907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1ca8e-7f79-4db9-95fa-7dbebaec265d service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.066067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1041/2091] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:55:11.116941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1ca8e-7f79-4db9-95fa-7dbebaec265d service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.118978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1051/2092] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:55:10 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:11.133560 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1ca8e-7f79-4db9-95fa-7dbebaec265d service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.234266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:11.251624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e212015-98a3-451c-94ba-2451301340a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.252760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e212015-98a3-451c-94ba-2451301340a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.253234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e212015-98a3-451c-94ba-2451301340a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.253611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e212015-98a3-451c-94ba-2451301340a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.254788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e212015-98a3-451c-94ba-2451301340a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.258083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e212015-98a3-451c-94ba-2451301340a4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.268313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.270651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:11.298706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2329ae9-37f0-471d-8392-6bec6955495f admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.304352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1042/2093] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:55:11 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:55:11.305610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e212015-98a3-451c-94ba-2451301340a4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.306528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.307469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.307757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.314632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] PATH_INFO: `/v3/projects/b2b9cfafc0fa4867a7b842745102a5a8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.318739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b9cfafc0fa4867a7b842745102a5a8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.320819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.326859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e212015-98a3-451c-94ba-2451301340a4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.350646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659286b-8a6b-4ec4-8742-82f51bb27ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.351480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:11.351608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659286b-8a6b-4ec4-8742-82f51bb27ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.351870 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659286b-8a6b-4ec4-8742-82f51bb27ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.352133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659286b-8a6b-4ec4-8742-82f51bb27ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.353338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659286b-8a6b-4ec4-8742-82f51bb27ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.358514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659286b-8a6b-4ec4-8742-82f51bb27ce8 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.393386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1043/2094] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:11.394583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659286b-8a6b-4ec4-8742-82f51bb27ce8 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.412167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659286b-8a6b-4ec4-8742-82f51bb27ce8 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.421355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b9cfafc0fa4867a7b842745102a5a8', 'request_id': 'req-ff9d2aac-54a5-40a5-a099-1fa053164aef'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.434321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b9cfafc0fa4867a7b842745102a5a8', 'request_id': 'req-ff9d2aac-54a5-40a5-a099-1fa053164aef'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.444617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b9cfafc0fa4867a7b842745102a5a8', 'request_id': 'req-ff9d2aac-54a5-40a5-a099-1fa053164aef'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.460085 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] The token cache is being invalidate because project b2b9cfafc0fa4867a7b842745102a5a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:11.460383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.461100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.463100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1052/2095] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:11 2022] DELETE /identity/v3/projects/b2b9cfafc0fa4867a7b842745102a5a8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:11.467438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.469220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:11.513468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.514321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.514733 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.515088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.515940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.518382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.532295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1044/2096] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:11.537698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.545349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:11.565339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20124dc7-1ac6-4114-a110-a79711c8343c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.571983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.576248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.576654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.577147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] PATH_INFO: `/v3/projects/fe5254586293427da7a03e7ef1baa6c0/users/f13a350be5a142a28281232d4fc7cd56/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.587706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d2aac-54a5-40a5-a099-1fa053164aef admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.595001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.595961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.596270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.596570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] PATH_INFO: `/v3/users/2c1e902ee2d0420f8709d93432c47ef7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.598746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c1e902ee2d0420f8709d93432c47ef7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.600888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.612942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5254586293427da7a03e7ef1baa6c0, user_id=f13a350be5a142a28281232d4fc7cd56, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.614391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.642149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1045/2097] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:11 2022] PUT /identity/v3/projects/fe5254586293427da7a03e7ef1baa6c0/users/f13a350be5a142a28281232d4fc7cd56/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:11.642925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.647636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:11.698193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c1e902ee2d0420f8709d93432c47ef7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.715708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c1e902ee2d0420f8709d93432c47ef7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.728123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c1e902ee2d0420f8709d93432c47ef7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.728444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c1e902ee2d0420f8709d93432c47ef7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.744132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f30a4-5a93-4c6e-a761-eb884f37bcf1 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:11.745000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-83506174-89b3-4d44-9fd3-59236164c923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c1e902ee2d0420f8709d93432c47ef7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.749121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:11.750237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.750535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.750926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] PATH_INFO: `/v3/projects/325f8918f0f9436bb72569a891654da2` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.751522 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1053/2098] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:11 2022] DELETE /identity/v3/users/2c1e902ee2d0420f8709d93432c47ef7 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:55:11.754551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c09cb-87e0-49f3-9b4d-2f4f6a5bfd7b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:11.754551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c09cb-87e0-49f3-9b4d-2f4f6a5bfd7b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:11.754551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c09cb-87e0-49f3-9b4d-2f4f6a5bfd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:11.755553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=325f8918f0f9436bb72569a891654da2)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:11.758176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:11.804857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ee4c09cb-87e0-49f3-9b4d-2f4f6a5bfd7b None None] MFA Rules not processed for user `f13a350be5a142a28281232d4fc7cd56`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:11.806606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ee4c09cb-87e0-49f3-9b4d-2f4f6a5bfd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:11.813333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:11.835187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:11.837495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '325f8918f0f9436bb72569a891654da2', 'request_id': 'req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.857423 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '325f8918f0f9436bb72569a891654da2', 'request_id': 'req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.878151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '325f8918f0f9436bb72569a891654da2', 'request_id': 'req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.899693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] The token cache is being invalidate because project 325f8918f0f9436bb72569a891654da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:11.900218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.901232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:11.903175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1046/2099] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:11 2022] DELETE /identity/v3/projects/325f8918f0f9436bb72569a891654da2 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:11.907918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.914850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:11.917192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1054/2100] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:11.930611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c09cb-87e0-49f3-9b4d-2f4f6a5bfd7b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:11.932324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ee4c09cb-87e0-49f3-9b4d-2f4f6a5bfd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.010759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccddc2-1fb9-4e4a-a0c1-6805bee03424 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.015747 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.026064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.029096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.029096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] PATH_INFO: `/v3/users/a3c5275a337849a7ae9b849492e8ec06` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.031536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c5275a337849a7ae9b849492e8ec06)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.038749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.041671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c09cb-87e0-49f3-9b4d-2f4f6a5bfd7b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.049930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.052295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.052585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.052869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.056757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.080900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.082711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.094524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1055/2101] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:12.150145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c5275a337849a7ae9b849492e8ec06'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.170862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.172331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.173174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c5275a337849a7ae9b849492e8ec06'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.189436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c5275a337849a7ae9b849492e8ec06'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.202769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c5275a337849a7ae9b849492e8ec06'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.203524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bb8982bb-e0e4-4a87-94f7-7a2555112f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c5275a337849a7ae9b849492e8ec06'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.210476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1047/2102] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:11 2022] DELETE /identity/v3/users/a3c5275a337849a7ae9b849492e8ec06 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:12.216775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd89ca1-ead4-48c0-b186-993e1a357ed6 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.299093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.300145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.300281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.300494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] PATH_INFO: `/v3/projects/06c96fcf71694e94b31403913afd9446` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.302124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c96fcf71694e94b31403913afd9446)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.304281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.370388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c96fcf71694e94b31403913afd9446', 'request_id': 'req-3912a264-027c-47af-b102-9db668ffbee9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.372990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:12.379012 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c96fcf71694e94b31403913afd9446', 'request_id': 'req-3912a264-027c-47af-b102-9db668ffbee9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.390760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c96fcf71694e94b31403913afd9446', 'request_id': 'req-3912a264-027c-47af-b102-9db668ffbee9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.407587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] The token cache is being invalidate because project 06c96fcf71694e94b31403913afd9446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:12.407902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.408701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.411343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1056/2103] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:12 2022] DELETE /identity/v3/projects/06c96fcf71694e94b31403913afd9446 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:55:12.416554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd06c-d469-481a-b8f4-03af4909024b None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.417119 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd06c-d469-481a-b8f4-03af4909024b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.417231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dd06c-d469-481a-b8f4-03af4909024b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.441889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0f7dd06c-d469-481a-b8f4-03af4909024b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:12.443137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0f7dd06c-d469-481a-b8f4-03af4909024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.537403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:12.561446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1048/2104] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:12.569231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.570693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.573472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356f681-fe80-49ac-b4d8-e43b6adf67af None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.573765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356f681-fe80-49ac-b4d8-e43b6adf67af None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.574062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356f681-fe80-49ac-b4d8-e43b6adf67af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.586916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912a264-027c-47af-b102-9db668ffbee9 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.603178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.604592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-4356f681-fe80-49ac-b4d8-e43b6adf67af None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:12.605147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.606028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-4356f681-fe80-49ac-b4d8-e43b6adf67af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.606190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.606454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] PATH_INFO: `/v3/projects/f341abba58b74b9facf5e1aa42d0f933` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.607921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f341abba58b74b9facf5e1aa42d0f933)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.610315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.616066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1049/2105] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:12.621751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356f681-fe80-49ac-b4d8-e43b6adf67af None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.631428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356f681-fe80-49ac-b4d8-e43b6adf67af None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.638862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.639640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.639992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.640340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.641142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:55:12.641454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.643105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.657830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1050/2106] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:55:12.665079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.677950 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673f5fc-65ed-4db7-b633-2393c25b9160 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.680334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f341abba58b74b9facf5e1aa42d0f933', 'request_id': 'req-347bf8a1-1fd3-44c3-a1c6-86d12082b257'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.688463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f341abba58b74b9facf5e1aa42d0f933', 'request_id': 'req-347bf8a1-1fd3-44c3-a1c6-86d12082b257'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.695896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f341abba58b74b9facf5e1aa42d0f933', 'request_id': 'req-347bf8a1-1fd3-44c3-a1c6-86d12082b257'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.709260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] The token cache is being invalidate because project f341abba58b74b9facf5e1aa42d0f933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:12.709825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.710773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-347bf8a1-1fd3-44c3-a1c6-86d12082b257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:12.712684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1057/2107] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:12 2022] DELETE /identity/v3/projects/f341abba58b74b9facf5e1aa42d0f933 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:12.718582 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157faeb-d252-43e8-ad24-30810251c898 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.718927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157faeb-d252-43e8-ad24-30810251c898 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.719279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157faeb-d252-43e8-ad24-30810251c898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.739652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-8157faeb-d252-43e8-ad24-30810251c898 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:12.740873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8157faeb-d252-43e8-ad24-30810251c898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.755744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.756464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.756751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.757037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.757837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.759974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.789833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1051/2108] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:55:12 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:55:12.792854 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.794519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.806359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1058/2109] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:12.814729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfae0ee-634d-43a1-8972-c16268b15d23 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.814931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfae0ee-634d-43a1-8972-c16268b15d23 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.815323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfae0ee-634d-43a1-8972-c16268b15d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.835026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-9cfae0ee-634d-43a1-8972-c16268b15d23 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:12.836259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9cfae0ee-634d-43a1-8972-c16268b15d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:12.879013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d9bdc-600f-4883-a9c9-d225460bf1cd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.888624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06462a71-3c5e-44af-bb2c-43eb04c87c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.889404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06462a71-3c5e-44af-bb2c-43eb04c87c87 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.889748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06462a71-3c5e-44af-bb2c-43eb04c87c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.890827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06462a71-3c5e-44af-bb2c-43eb04c87c87 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.891685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06462a71-3c5e-44af-bb2c-43eb04c87c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.893836 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06462a71-3c5e-44af-bb2c-43eb04c87c87 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.900409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1059/2110] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:55:12.904427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfae0ee-634d-43a1-8972-c16268b15d23 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.911323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:12.915449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfae0ee-634d-43a1-8972-c16268b15d23 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.917586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1052/2111] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:55:12 2022] POST /identity/v3/users => generated 286 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:55:12.922018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.922272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06462a71-3c5e-44af-bb2c-43eb04c87c87 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.923194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.923613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.924331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.925445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:55:12.925858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.929267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.937030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06462a71-3c5e-44af-bb2c-43eb04c87c87 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.943355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1060/2112] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 15:55:12.945692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a69f8-65de-4b23-add2-f8fdae38f806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.946613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a69f8-65de-4b23-add2-f8fdae38f806 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.946891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a69f8-65de-4b23-add2-f8fdae38f806 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.947216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a69f8-65de-4b23-add2-f8fdae38f806 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.948182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a69f8-65de-4b23-add2-f8fdae38f806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.950068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a69f8-65de-4b23-add2-f8fdae38f806 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.950902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.962516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1053/2113] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:55:12.967428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a69f8-65de-4b23-add2-f8fdae38f806 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:12.968910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667f8e1-a3f9-4d47-9b65-a088ea2abf95 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.975297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d2d1d-6682-45a6-a0ec-3fd3d6242bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.976056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2d1d-6682-45a6-a0ec-3fd3d6242bbc admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.976329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2d1d-6682-45a6-a0ec-3fd3d6242bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.976624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2d1d-6682-45a6-a0ec-3fd3d6242bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.977393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d2d1d-6682-45a6-a0ec-3fd3d6242bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.979725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a69f8-65de-4b23-add2-f8fdae38f806 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:12.980275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d2d1d-6682-45a6-a0ec-3fd3d6242bbc admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:12.984885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc0bf0-52ba-4203-9938-968912ec405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:12.985953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0bf0-52ba-4203-9938-968912ec405b admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:12.986234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0bf0-52ba-4203-9938-968912ec405b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:12.986495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc0bf0-52ba-4203-9938-968912ec405b admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:12.987685 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc0bf0-52ba-4203-9938-968912ec405b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:12.989857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc0bf0-52ba-4203-9938-968912ec405b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:13.021178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1054/2114] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:55:13.025110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc0bf0-52ba-4203-9938-968912ec405b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:13.025548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1061/2115] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:55:12 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:13.029942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d2d1d-6682-45a6-a0ec-3fd3d6242bbc admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:13.043790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc0bf0-52ba-4203-9938-968912ec405b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:13.072985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:13.110790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2c177-23a8-4db1-9fa0-e38459d8b24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:13.111714 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2c177-23a8-4db1-9fa0-e38459d8b24c admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.112142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2c177-23a8-4db1-9fa0-e38459d8b24c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.112467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2c177-23a8-4db1-9fa0-e38459d8b24c admin admin] PATH_INFO: `/v3/projects/06db592518d94f498a57c52121e32ca0/users/d7c5e4602e9f4845a1a498b702c567d5/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.116853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d2d1d-6682-45a6-a0ec-3fd3d6242bbc admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:13.121845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a629c33-8e7c-422f-9216-e2668e00bb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:13.122500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a629c33-8e7c-422f-9216-e2668e00bb1f admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.122784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a629c33-8e7c-422f-9216-e2668e00bb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.123059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a629c33-8e7c-422f-9216-e2668e00bb1f admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.123762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a629c33-8e7c-422f-9216-e2668e00bb1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:13.125699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a629c33-8e7c-422f-9216-e2668e00bb1f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:13.133115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2c177-23a8-4db1-9fa0-e38459d8b24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06db592518d94f498a57c52121e32ca0, user_id=d7c5e4602e9f4845a1a498b702c567d5, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:13.138127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2c177-23a8-4db1-9fa0-e38459d8b24c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:13.159711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1055/2116] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:13 2022] PUT /identity/v3/projects/06db592518d94f498a57c52121e32ca0/users/d7c5e4602e9f4845a1a498b702c567d5/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:55:13.159927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1062/2117] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:55:13 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:13.165131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2c177-23a8-4db1-9fa0-e38459d8b24c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:13.165765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3aa9fe-13ff-4bf8-b9f8-464e55ed4a84 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.166161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3aa9fe-13ff-4bf8-b9f8-464e55ed4a84 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.166452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3aa9fe-13ff-4bf8-b9f8-464e55ed4a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.197831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5e3aa9fe-13ff-4bf8-b9f8-464e55ed4a84 None None] MFA Rules not processed for user `d7c5e4602e9f4845a1a498b702c567d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:13.199194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5e3aa9fe-13ff-4bf8-b9f8-464e55ed4a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:13.255998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2c177-23a8-4db1-9fa0-e38459d8b24c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:13.261326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23ea9-aade-4d51-9ea9-e769f57e6aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:13.262053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23ea9-aade-4d51-9ea9-e769f57e6aaa admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.262345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23ea9-aade-4d51-9ea9-e769f57e6aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.262619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23ea9-aade-4d51-9ea9-e769f57e6aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.263345 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a23ea9-aade-4d51-9ea9-e769f57e6aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:13.264971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a23ea9-aade-4d51-9ea9-e769f57e6aaa admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:13.279521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1056/2118] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:13.279720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1063/2119] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:13.280574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23ea9-aade-4d51-9ea9-e769f57e6aaa admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:13.291059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23ea9-aade-4d51-9ea9-e769f57e6aaa admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:13.291534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3aa9fe-13ff-4bf8-b9f8-464e55ed4a84 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:13.293006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5e3aa9fe-13ff-4bf8-b9f8-464e55ed4a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:13.297875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0baa7b-a226-493e-9a3b-ca0520e2284f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:13.298831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0baa7b-a226-493e-9a3b-ca0520e2284f admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.299236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0baa7b-a226-493e-9a3b-ca0520e2284f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.299632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0baa7b-a226-493e-9a3b-ca0520e2284f admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.300733 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0baa7b-a226-493e-9a3b-ca0520e2284f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:13.303274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0baa7b-a226-493e-9a3b-ca0520e2284f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:13.322311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1057/2120] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:13.330352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0baa7b-a226-493e-9a3b-ca0520e2284f admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:13.342898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0baa7b-a226-493e-9a3b-ca0520e2284f admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:13.349206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235d368-78da-47d9-adb0-be4262c4eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:13.350177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d368-78da-47d9-adb0-be4262c4eb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.350457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d368-78da-47d9-adb0-be4262c4eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.350762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235d368-78da-47d9-adb0-be4262c4eb96 admin admin] PATH_INFO: `/v3/projects/a1230bebeb364d4599cfa609414ad5eb/users/2eac4600e37949a3b86eb08436217b58/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.367955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:13.368311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235d368-78da-47d9-adb0-be4262c4eb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1230bebeb364d4599cfa609414ad5eb, user_id=2eac4600e37949a3b86eb08436217b58, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:13.371767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235d368-78da-47d9-adb0-be4262c4eb96 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:13.372783 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3aa9fe-13ff-4bf8-b9f8-464e55ed4a84 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:13.377979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:13.378666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.379001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.379685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.387166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1058/2121] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:13 2022] PUT /identity/v3/projects/a1230bebeb364d4599cfa609414ad5eb/users/2eac4600e37949a3b86eb08436217b58/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:13.389200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:13.390723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:13.399267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679e40-cef0-4419-bad8-635251bfc797 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.399556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679e40-cef0-4419-bad8-635251bfc797 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.399865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679e40-cef0-4419-bad8-635251bfc797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.400937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1064/2122] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:55:13.430332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-68679e40-cef0-4419-bad8-635251bfc797 None None] MFA Rules not processed for user `2eac4600e37949a3b86eb08436217b58`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:13.431810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-68679e40-cef0-4419-bad8-635251bfc797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:13.513242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1059/2123] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:13.523414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:13.524941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:13.591271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516a3a6-1521-40ec-b86c-ad6092d031a0 service glance] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:13.654937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b80f0-0041-439c-855f-554e9d173517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:13.655646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b80f0-0041-439c-855f-554e9d173517 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:13.655907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b80f0-0041-439c-855f-554e9d173517 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:13.656178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b80f0-0041-439c-855f-554e9d173517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:13.666514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b80f0-0041-439c-855f-554e9d173517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:13.667986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b80f0-0041-439c-855f-554e9d173517 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:13.677743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1065/2124] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:23.504640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:23.673304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=50 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:46.168625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68679e40-cef0-4419-bad8-635251bfc797 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:46.242988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68679e40-cef0-4419-bad8-635251bfc797 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:46.249403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:46.250348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:46.250625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:46.250895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] PATH_INFO: `/v3/users/2eac4600e37949a3b86eb08436217b58` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:46.252233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eac4600e37949a3b86eb08436217b58)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:46.254161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:46.307507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eac4600e37949a3b86eb08436217b58'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:46.320078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eac4600e37949a3b86eb08436217b58'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:46.328302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eac4600e37949a3b86eb08436217b58'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:46.335405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eac4600e37949a3b86eb08436217b58'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:46.336049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eac4600e37949a3b86eb08436217b58'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:46.342693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1060/2125] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:46 2022] DELETE /identity/v3/users/2eac4600e37949a3b86eb08436217b58 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:46.347675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b35e68-854d-4b9f-88d1-6e6ae88c588d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:46.348193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b35e68-854d-4b9f-88d1-6e6ae88c588d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:46.348292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b35e68-854d-4b9f-88d1-6e6ae88c588d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:46.368372 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-08b35e68-854d-4b9f-88d1-6e6ae88c588d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:46.369826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-08b35e68-854d-4b9f-88d1-6e6ae88c588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:46.432178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1066/2126] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:46.442315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:46.443739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:46.502209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b80aef-469b-4cef-baed-c033280b7cd2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:46.506015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30d539-1026-4a08-b3bf-056e0abad063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:46.506659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d539-1026-4a08-b3bf-056e0abad063 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:46.506941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d539-1026-4a08-b3bf-056e0abad063 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:46.507212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30d539-1026-4a08-b3bf-056e0abad063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:46.518050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30d539-1026-4a08-b3bf-056e0abad063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:46.519499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30d539-1026-4a08-b3bf-056e0abad063 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:46.529552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1061/2127] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:55:47.059171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b35e68-854d-4b9f-88d1-6e6ae88c588d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:47.061577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-08b35e68-854d-4b9f-88d1-6e6ae88c588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:47.075978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b35e68-854d-4b9f-88d1-6e6ae88c588d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:47.082072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:47.083108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.083439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.083847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] PATH_INFO: `/v3/projects/a1230bebeb364d4599cfa609414ad5eb` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.085367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1230bebeb364d4599cfa609414ad5eb)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:47.087404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:47.144009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1230bebeb364d4599cfa609414ad5eb', 'request_id': 'req-b406df83-9183-447e-9910-54e7ce86c92f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:47.153038 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1230bebeb364d4599cfa609414ad5eb', 'request_id': 'req-b406df83-9183-447e-9910-54e7ce86c92f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:47.159795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1230bebeb364d4599cfa609414ad5eb', 'request_id': 'req-b406df83-9183-447e-9910-54e7ce86c92f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:47.172024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] The token cache is being invalidate because project a1230bebeb364d4599cfa609414ad5eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:47.172422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:47.173124 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b406df83-9183-447e-9910-54e7ce86c92f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:47.178857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1067/2128] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:47 2022] DELETE /identity/v3/projects/a1230bebeb364d4599cfa609414ad5eb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:47.182202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fe19-100e-4ea2-9dc1-22590a603ed1 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.182202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fe19-100e-4ea2-9dc1-22590a603ed1 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.182730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fe19-100e-4ea2-9dc1-22590a603ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.205867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-fe02fe19-100e-4ea2-9dc1-22590a603ed1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:47.207354 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fe02fe19-100e-4ea2-9dc1-22590a603ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:47.233897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:47.297164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1062/2129] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:55:47.305166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c749-cf61-49a3-99be-cc526ede7a1c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.305461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c749-cf61-49a3-99be-cc526ede7a1c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.305816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c749-cf61-49a3-99be-cc526ede7a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.326595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-0e95c749-cf61-49a3-99be-cc526ede7a1c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:47.327811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0e95c749-cf61-49a3-99be-cc526ede7a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:47.336666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1068/2130] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:55:47.341289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02fe19-100e-4ea2-9dc1-22590a603ed1 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:47.351746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02fe19-100e-4ea2-9dc1-22590a603ed1 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:47.357175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:47.357824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.358140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.358412 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.359120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:55:47.359434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:47.361136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:47.374593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1063/2131] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:55:47.379191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95c749-cf61-49a3-99be-cc526ede7a1c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:47.389037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95c749-cf61-49a3-99be-cc526ede7a1c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:47.394406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abc155-ae47-47b3-b785-09da2fc15639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:47.395014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abc155-ae47-47b3-b785-09da2fc15639 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.395298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abc155-ae47-47b3-b785-09da2fc15639 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.395576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abc155-ae47-47b3-b785-09da2fc15639 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.396281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abc155-ae47-47b3-b785-09da2fc15639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:47.398264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abc155-ae47-47b3-b785-09da2fc15639 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:47.429854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1069/2132] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:55:47 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:47.431086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:47.442474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728f412-d36a-458f-945f-e7f810bdf063 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:47.513723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9c42f-19c8-43fb-a30f-d53d98c82265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:47.514222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9c42f-19c8-43fb-a30f-d53d98c82265 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.514502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9c42f-19c8-43fb-a30f-d53d98c82265 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.514776 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9c42f-19c8-43fb-a30f-d53d98c82265 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.515601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9c42f-19c8-43fb-a30f-d53d98c82265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:47.517419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9c42f-19c8-43fb-a30f-d53d98c82265 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:47.539374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1064/2133] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:55:47 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:47.544382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abc155-ae47-47b3-b785-09da2fc15639 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:47.560573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abc155-ae47-47b3-b785-09da2fc15639 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:47.566215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaac30-7979-48a2-ae6b-8ddfe695abe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:47.566886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaac30-7979-48a2-ae6b-8ddfe695abe8 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.567165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaac30-7979-48a2-ae6b-8ddfe695abe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.567437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaac30-7979-48a2-ae6b-8ddfe695abe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.568353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaac30-7979-48a2-ae6b-8ddfe695abe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:47.570157 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaac30-7979-48a2-ae6b-8ddfe695abe8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:47.581919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1070/2134] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:55:47.586049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9c42f-19c8-43fb-a30f-d53d98c82265 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:47.597896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9c42f-19c8-43fb-a30f-d53d98c82265 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:47.603721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a12c85-c6cf-4454-90e5-398f69777b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:47.604462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a12c85-c6cf-4454-90e5-398f69777b18 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.604751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a12c85-c6cf-4454-90e5-398f69777b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.605064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a12c85-c6cf-4454-90e5-398f69777b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.605839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a12c85-c6cf-4454-90e5-398f69777b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:47.607460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a12c85-c6cf-4454-90e5-398f69777b18 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:47.617938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1065/2135] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:47.622655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaac30-7979-48a2-ae6b-8ddfe695abe8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:47.632767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaac30-7979-48a2-ae6b-8ddfe695abe8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:47.639142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:47.640068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.640336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.640609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] PATH_INFO: `/v3/projects/d9e05541a50349b29f84813567782a04/users/47b9e003cff740cfa3ef413177218f2c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.657054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e05541a50349b29f84813567782a04, user_id=47b9e003cff740cfa3ef413177218f2c, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:47.659677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:47.674293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1071/2136] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:47 2022] PUT /identity/v3/projects/d9e05541a50349b29f84813567782a04/users/47b9e003cff740cfa3ef413177218f2c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:55:47.684629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612020b-ed7b-45d8-bbf0-2968e9d1722c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.685000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612020b-ed7b-45d8-bbf0-2968e9d1722c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.685350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612020b-ed7b-45d8-bbf0-2968e9d1722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.714755 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-6612020b-ed7b-45d8-bbf0-2968e9d1722c None None] MFA Rules not processed for user `47b9e003cff740cfa3ef413177218f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:47.716025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6612020b-ed7b-45d8-bbf0-2968e9d1722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:47.782972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1066/2137] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:47.793565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:47.795333 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:47.823848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:47.866170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7b95-181d-45c0-b44c-a99667ebd744 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:47.870150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1757c-13fc-462c-868b-74cd0e8df5be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:47.871073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1757c-13fc-462c-868b-74cd0e8df5be service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:47.871073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1757c-13fc-462c-868b-74cd0e8df5be service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:47.871355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1757c-13fc-462c-868b-74cd0e8df5be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:47.880346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1757c-13fc-462c-868b-74cd0e8df5be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:47.881824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1757c-13fc-462c-868b-74cd0e8df5be service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:47.891925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1072/2138] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:50.907016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612020b-ed7b-45d8-bbf0-2968e9d1722c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:50.908465 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6612020b-ed7b-45d8-bbf0-2968e9d1722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:50.934607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:50.998603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612020b-ed7b-45d8-bbf0-2968e9d1722c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:51.006962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:51.008582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.009089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.010887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] PATH_INFO: `/v3/users/23031b94dc7f47bf8272b266cbe18a8c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.012605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23031b94dc7f47bf8272b266cbe18a8c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:51.014728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:51.101892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23031b94dc7f47bf8272b266cbe18a8c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.119505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23031b94dc7f47bf8272b266cbe18a8c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.127340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23031b94dc7f47bf8272b266cbe18a8c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.134887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23031b94dc7f47bf8272b266cbe18a8c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.135555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-edaa0eb3-a2fc-4daa-bdf8-9678ed4b5e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23031b94dc7f47bf8272b266cbe18a8c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.146032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1067/2139] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:50 2022] DELETE /identity/v3/users/23031b94dc7f47bf8272b266cbe18a8c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:51.331379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1757c-13fc-462c-868b-74cd0e8df5be service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:51.343578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1757c-13fc-462c-868b-74cd0e8df5be service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:51.404322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:51.405417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.405826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.406193 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] PATH_INFO: `/v3/projects/707b41a2e0ce4a8abc4e26c88ce94267` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.407971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=707b41a2e0ce4a8abc4e26c88ce94267)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:51.410772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:51.461233 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707b41a2e0ce4a8abc4e26c88ce94267', 'request_id': 'req-764a688d-ac47-4aa1-92bc-06ae891ffb8f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.469716 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707b41a2e0ce4a8abc4e26c88ce94267', 'request_id': 'req-764a688d-ac47-4aa1-92bc-06ae891ffb8f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.476757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707b41a2e0ce4a8abc4e26c88ce94267', 'request_id': 'req-764a688d-ac47-4aa1-92bc-06ae891ffb8f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.488001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] The token cache is being invalidate because project 707b41a2e0ce4a8abc4e26c88ce94267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:51.488288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.488939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-764a688d-ac47-4aa1-92bc-06ae891ffb8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:51.490669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1073/2140] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:51 2022] DELETE /identity/v3/projects/707b41a2e0ce4a8abc4e26c88ce94267 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:51.496245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf765b-280d-4333-8fe6-1ba3f16eead0 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.496536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf765b-280d-4333-8fe6-1ba3f16eead0 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.496814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf765b-280d-4333-8fe6-1ba3f16eead0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.516073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8dcf765b-280d-4333-8fe6-1ba3f16eead0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:51.517278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8dcf765b-280d-4333-8fe6-1ba3f16eead0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:51.579850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1068/2141] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:51.592640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804ccbe-4316-455c-9cd2-38d4a35088b3 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.592885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804ccbe-4316-455c-9cd2-38d4a35088b3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.593112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804ccbe-4316-455c-9cd2-38d4a35088b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.613648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-4804ccbe-4316-455c-9cd2-38d4a35088b3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:51.615052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4804ccbe-4316-455c-9cd2-38d4a35088b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:51.627918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1074/2142] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:51.629055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf765b-280d-4333-8fe6-1ba3f16eead0 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:51.639446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf765b-280d-4333-8fe6-1ba3f16eead0 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:51.644397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:51.645066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.645346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.645622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.646388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:55:51.646693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:51.648298 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:51.660665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1069/2143] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:55:51.665125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804ccbe-4316-455c-9cd2-38d4a35088b3 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:51.674886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804ccbe-4316-455c-9cd2-38d4a35088b3 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:51.680602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8271-4208-4a7f-ae79-d4374eb0e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:51.681305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8271-4208-4a7f-ae79-d4374eb0e9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.681668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8271-4208-4a7f-ae79-d4374eb0e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.681982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8271-4208-4a7f-ae79-d4374eb0e9d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.682695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac8271-4208-4a7f-ae79-d4374eb0e9d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:51.684665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac8271-4208-4a7f-ae79-d4374eb0e9d5 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:51.711297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1075/2144] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:55:51 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:51.714914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:51.724954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2993-d122-49c3-a835-ced89a87d4c7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:51.747638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:51.790167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc0f75-b2cb-4558-887b-21700d88f14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:51.790845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc0f75-b2cb-4558-887b-21700d88f14c admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.791155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc0f75-b2cb-4558-887b-21700d88f14c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.791500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc0f75-b2cb-4558-887b-21700d88f14c admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.792209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc0f75-b2cb-4558-887b-21700d88f14c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:51.794162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbc0f75-b2cb-4558-887b-21700d88f14c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:51.815769 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1070/2145] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:55:51 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:55:51.820122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8271-4208-4a7f-ae79-d4374eb0e9d5 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:51.831498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8271-4208-4a7f-ae79-d4374eb0e9d5 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:51.836818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8066cb-7fd2-4845-9923-67a42847c810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:51.837881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8066cb-7fd2-4845-9923-67a42847c810 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.838168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8066cb-7fd2-4845-9923-67a42847c810 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.838445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8066cb-7fd2-4845-9923-67a42847c810 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.839178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8066cb-7fd2-4845-9923-67a42847c810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:51.840773 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8066cb-7fd2-4845-9923-67a42847c810 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:51.851731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1076/2146] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:51.861275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc0f75-b2cb-4558-887b-21700d88f14c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:51.872288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc0f75-b2cb-4558-887b-21700d88f14c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:51.878132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba3f5a-f316-4e22-b688-8ff4360ab7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:51.878780 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba3f5a-f316-4e22-b688-8ff4360ab7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.879053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba3f5a-f316-4e22-b688-8ff4360ab7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.879326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba3f5a-f316-4e22-b688-8ff4360ab7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.880051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba3f5a-f316-4e22-b688-8ff4360ab7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:51.881640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba3f5a-f316-4e22-b688-8ff4360ab7a0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:51.893855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1071/2147] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:55:51.898496 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8066cb-7fd2-4845-9923-67a42847c810 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:51.909862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8066cb-7fd2-4845-9923-67a42847c810 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:51.915037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:51.916029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.916306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.916574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] PATH_INFO: `/v3/projects/8818885318b54ef6858e754d2bc4587c/users/01571ac0b01845f78192269cacb23644/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.929927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:51.935904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8818885318b54ef6858e754d2bc4587c, user_id=01571ac0b01845f78192269cacb23644, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:51.939808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:51.954797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1077/2148] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:55:51 2022] PUT /identity/v3/projects/8818885318b54ef6858e754d2bc4587c/users/01571ac0b01845f78192269cacb23644/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:55:51.960108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc878d6-9595-4d6b-b082-0151d9b98186 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:51.960611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc878d6-9595-4d6b-b082-0151d9b98186 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:51.960710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc878d6-9595-4d6b-b082-0151d9b98186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:51.980288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0fc878d6-9595-4d6b-b082-0151d9b98186 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:51.981476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0fc878d6-9595-4d6b-b082-0151d9b98186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:52.046255 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1072/2149] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:55:52.055874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:52.058455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:52.129746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff7ef0-1205-45a9-a2e4-bcdf0d6d082f admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:52.136764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e59e68-c4db-4cba-91f0-bce935c56490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:52.137408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e59e68-c4db-4cba-91f0-bce935c56490 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:52.137542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e59e68-c4db-4cba-91f0-bce935c56490 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:52.137828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e59e68-c4db-4cba-91f0-bce935c56490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:52.149614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e59e68-c4db-4cba-91f0-bce935c56490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:52.152344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e59e68-c4db-4cba-91f0-bce935c56490 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:52.163516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1078/2150] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:56.197891 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5270-0bd9-41f2-859f-d96545616bbc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:56.198368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5270-0bd9-41f2-859f-d96545616bbc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:56.198368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf5270-0bd9-41f2-859f-d96545616bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:56.247539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7bdf5270-0bd9-41f2-859f-d96545616bbc None None] MFA Rules not processed for user `01571ac0b01845f78192269cacb23644`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:56.249139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7bdf5270-0bd9-41f2-859f-d96545616bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:56.418520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1073/2151] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:56.423109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ef65-fd2f-4b74-bc72-ce1f6b055385 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:56.423608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ef65-fd2f-4b74-bc72-ce1f6b055385 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:56.423725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600ef65-fd2f-4b74-bc72-ce1f6b055385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:56.444360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5600ef65-fd2f-4b74-bc72-ce1f6b055385 None None] MFA Rules not processed for user `01571ac0b01845f78192269cacb23644`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:56.446057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5600ef65-fd2f-4b74-bc72-ce1f6b055385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:56.466705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1079/2152] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:56.478971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf5270-0bd9-41f2-859f-d96545616bbc None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:56.483004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7bdf5270-0bd9-41f2-859f-d96545616bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:56.489675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:56.608029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf5270-0bd9-41f2-859f-d96545616bbc None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:56.613985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:56.614664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:56.614948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:56.615227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:56.625054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:56.626727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:56.637108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1074/2153] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:57.363226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0898-3085-428d-8fb2-a3c2533689fa None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:57.363523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0898-3085-428d-8fb2-a3c2533689fa None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:57.363804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0898-3085-428d-8fb2-a3c2533689fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:57.386801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ea6e0898-3085-428d-8fb2-a3c2533689fa None None] MFA Rules not processed for user `01571ac0b01845f78192269cacb23644`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:57.387366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ea6e0898-3085-428d-8fb2-a3c2533689fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:57.410015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1080/2154] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:57.495796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:57.497625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:57.589359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a935f4-33c3-48b8-a75b-aa0887fb94d9 service nova] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:57.598020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:57.598763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:57.598998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:57.599165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] PATH_INFO: `/v3/users/47b9e003cff740cfa3ef413177218f2c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:57.600831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b9e003cff740cfa3ef413177218f2c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:57.602952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:57.682690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b9e003cff740cfa3ef413177218f2c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:57.704756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b9e003cff740cfa3ef413177218f2c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:57.715250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b9e003cff740cfa3ef413177218f2c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:57.731485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b9e003cff740cfa3ef413177218f2c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:57.732134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b9e003cff740cfa3ef413177218f2c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:57.743249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1075/2155] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:55:57 2022] DELETE /identity/v3/users/47b9e003cff740cfa3ef413177218f2c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:55:57.753766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7d227-b964-4095-bf59-494edc6c4abb None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:57.753766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7d227-b964-4095-bf59-494edc6c4abb None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:57.753766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7d227-b964-4095-bf59-494edc6c4abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:57.793135 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-79a7d227-b964-4095-bf59-494edc6c4abb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:55:57.795026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-79a7d227-b964-4095-bf59-494edc6c4abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:57.887930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1081/2156] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:55:57.897035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:57.898759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:57.938992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:58.054461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8a4b-d189-4f88-ab67-784c062ef776 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:58.060527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:58.061321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:58.061671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:58.062052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:58.075430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:58.077453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:58.110217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1076/2157] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:55:58.646609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7d227-b964-4095-bf59-494edc6c4abb None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:55:58.648293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-79a7d227-b964-4095-bf59-494edc6c4abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:55:58.689356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7d227-b964-4095-bf59-494edc6c4abb None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:55:58.706143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:55:58.708134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:55:58.709039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:55:58.712849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] PATH_INFO: `/v3/projects/d9e05541a50349b29f84813567782a04` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:55:58.718210 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e05541a50349b29f84813567782a04)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:55:58.723416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:55:58.862117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e05541a50349b29f84813567782a04', 'request_id': 'req-52b62c20-e976-404d-b8ad-28eb6ba96527'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:58.885482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e05541a50349b29f84813567782a04', 'request_id': 'req-52b62c20-e976-404d-b8ad-28eb6ba96527'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:58.894315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e05541a50349b29f84813567782a04', 'request_id': 'req-52b62c20-e976-404d-b8ad-28eb6ba96527'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:58.902289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:55:58.914168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] The token cache is being invalidate because project d9e05541a50349b29f84813567782a04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:55:58.914589 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:58.915961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-52b62c20-e976-404d-b8ad-28eb6ba96527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:55:58.918989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1082/2158] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:55:58 2022] DELETE /identity/v3/projects/d9e05541a50349b29f84813567782a04 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:02.377972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:02.381811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:02.396488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038571cf-b552-4367-933d-352ee1a33282 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:02.468605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:02.469514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:02.469763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:02.470243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] PATH_INFO: `/v3/users/f13a350be5a142a28281232d4fc7cd56` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:02.471587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13a350be5a142a28281232d4fc7cd56)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:02.474002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:02.530714 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13a350be5a142a28281232d4fc7cd56'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:02.544031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13a350be5a142a28281232d4fc7cd56'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:02.555487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13a350be5a142a28281232d4fc7cd56'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:02.564640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13a350be5a142a28281232d4fc7cd56'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:02.565517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13a350be5a142a28281232d4fc7cd56'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:02.572944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1077/2159] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:56:02 2022] DELETE /identity/v3/users/f13a350be5a142a28281232d4fc7cd56 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:02.577499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfab5-da29-4255-8084-56a57afbe239 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:02.578052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfab5-da29-4255-8084-56a57afbe239 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:02.578158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cfab5-da29-4255-8084-56a57afbe239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:02.599986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-990cfab5-da29-4255-8084-56a57afbe239 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:02.601086 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-990cfab5-da29-4255-8084-56a57afbe239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:02.674937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1083/2160] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:02.685908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:02.687417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:02.707784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:02.753429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31ffcb-4fb8-4c80-b5dc-421368d27744 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:02.757200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0dabba-92c5-4e4d-943c-75432e50f2e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:02.757881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dabba-92c5-4e4d-943c-75432e50f2e9 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:02.758180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dabba-92c5-4e4d-943c-75432e50f2e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:02.759084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0dabba-92c5-4e4d-943c-75432e50f2e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:02.769861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0dabba-92c5-4e4d-943c-75432e50f2e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:02.771203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0dabba-92c5-4e4d-943c-75432e50f2e9 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:02.786864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1078/2161] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:03.140669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cfab5-da29-4255-8084-56a57afbe239 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:03.142242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-990cfab5-da29-4255-8084-56a57afbe239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:03.160649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cfab5-da29-4255-8084-56a57afbe239 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:03.168137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:03.168911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.169219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.169557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] PATH_INFO: `/v3/projects/fe5254586293427da7a03e7ef1baa6c0` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.170973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5254586293427da7a03e7ef1baa6c0)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:03.173089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:03.233446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5254586293427da7a03e7ef1baa6c0', 'request_id': 'req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:03.247141 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5254586293427da7a03e7ef1baa6c0', 'request_id': 'req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:03.254488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5254586293427da7a03e7ef1baa6c0', 'request_id': 'req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:03.267787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] The token cache is being invalidate because project fe5254586293427da7a03e7ef1baa6c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:56:03.268093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:03.268744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ee5e89a7-6700-4ddc-ae9a-c09bb7411da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:03.270730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1084/2162] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:56:03 2022] DELETE /identity/v3/projects/fe5254586293427da7a03e7ef1baa6c0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:03.275815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2d102-68ae-4f4c-acbc-74d109bfe405 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.276110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2d102-68ae-4f4c-acbc-74d109bfe405 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.276394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2d102-68ae-4f4c-acbc-74d109bfe405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.324682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ecf2d102-68ae-4f4c-acbc-74d109bfe405 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:03.326340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ecf2d102-68ae-4f4c-acbc-74d109bfe405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:03.397785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1079/2163] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:03.407042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caefa1-ec75-41b1-942e-9aff0d6f9d7d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.407410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caefa1-ec75-41b1-942e-9aff0d6f9d7d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.407875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caefa1-ec75-41b1-942e-9aff0d6f9d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.430810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-14caefa1-ec75-41b1-942e-9aff0d6f9d7d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:03.432054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-14caefa1-ec75-41b1-942e-9aff0d6f9d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:03.441465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1085/2164] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:03.446464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2d102-68ae-4f4c-acbc-74d109bfe405 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:03.459818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2d102-68ae-4f4c-acbc-74d109bfe405 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:03.465751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:03.466421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.466703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.466995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.467706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:56:03.468093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:03.470735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:03.487952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1080/2165] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:56:03.494247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caefa1-ec75-41b1-942e-9aff0d6f9d7d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:03.504549 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14caefa1-ec75-41b1-942e-9aff0d6f9d7d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:03.510152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bedc3-ceca-4200-8f2d-6dd9f52e6e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:03.510850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bedc3-ceca-4200-8f2d-6dd9f52e6e48 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.511129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bedc3-ceca-4200-8f2d-6dd9f52e6e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.511400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bedc3-ceca-4200-8f2d-6dd9f52e6e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.512099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bedc3-ceca-4200-8f2d-6dd9f52e6e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:03.514068 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bedc3-ceca-4200-8f2d-6dd9f52e6e48 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:03.544501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1086/2166] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:56:03 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:03.549118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:03.560853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcc238-8caf-4e58-84fd-79d5640dd3a6 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:03.623633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5b2f9-c894-4066-afd2-7f352df3c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:03.624307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b2f9-c894-4066-afd2-7f352df3c037 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.624586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b2f9-c894-4066-afd2-7f352df3c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.625058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5b2f9-c894-4066-afd2-7f352df3c037 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.625753 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5b2f9-c894-4066-afd2-7f352df3c037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:03.627870 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5b2f9-c894-4066-afd2-7f352df3c037 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:03.655645 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1081/2167] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:56:03 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:03.657874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bedc3-ceca-4200-8f2d-6dd9f52e6e48 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:03.673206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bedc3-ceca-4200-8f2d-6dd9f52e6e48 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:03.678601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8566-6a47-431e-b3a7-2a2ff848506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:03.679339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8566-6a47-431e-b3a7-2a2ff848506d admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.679561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8566-6a47-431e-b3a7-2a2ff848506d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.679833 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa8566-6a47-431e-b3a7-2a2ff848506d admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.680849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa8566-6a47-431e-b3a7-2a2ff848506d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:03.682705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baa8566-6a47-431e-b3a7-2a2ff848506d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:03.694565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1087/2168] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:03.699320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5b2f9-c894-4066-afd2-7f352df3c037 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:03.710722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5b2f9-c894-4066-afd2-7f352df3c037 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:03.716455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c8919-c0e9-4b4c-8b2e-bc15b238af57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:03.717140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c8919-c0e9-4b4c-8b2e-bc15b238af57 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.717440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c8919-c0e9-4b4c-8b2e-bc15b238af57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.717705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c8919-c0e9-4b4c-8b2e-bc15b238af57 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.718488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c8919-c0e9-4b4c-8b2e-bc15b238af57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:03.720105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c8919-c0e9-4b4c-8b2e-bc15b238af57 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:03.731518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1082/2169] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:03.735967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8566-6a47-431e-b3a7-2a2ff848506d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:03.747156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baa8566-6a47-431e-b3a7-2a2ff848506d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:03.752552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:03.753466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.753748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.754058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] PATH_INFO: `/v3/projects/4ed83119a201412e9edc9b3b10cd2a03/users/73d18de5ef504b1dbf78793ed678006c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.771042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed83119a201412e9edc9b3b10cd2a03, user_id=73d18de5ef504b1dbf78793ed678006c, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:03.773727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:03.786960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1088/2170] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:56:03 2022] PUT /identity/v3/projects/4ed83119a201412e9edc9b3b10cd2a03/users/73d18de5ef504b1dbf78793ed678006c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:03.797472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813f784-e085-46ad-8486-496c3b141114 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:03.797840 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813f784-e085-46ad-8486-496c3b141114 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:03.798060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813f784-e085-46ad-8486-496c3b141114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:03.819171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:03.835602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d813f784-e085-46ad-8486-496c3b141114 None None] MFA Rules not processed for user `73d18de5ef504b1dbf78793ed678006c`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:03.837283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d813f784-e085-46ad-8486-496c3b141114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:03.913488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1083/2171] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:03.934304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:03.938208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:03.969915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:04.020489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d9ce7-0d02-4eb6-bf8e-6e063e4057d4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:04.030657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92c410-ae09-4ee0-965e-22a850683f00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:04.031329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92c410-ae09-4ee0-965e-22a850683f00 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:04.031608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92c410-ae09-4ee0-965e-22a850683f00 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:04.031879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92c410-ae09-4ee0-965e-22a850683f00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:04.041580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92c410-ae09-4ee0-965e-22a850683f00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:04.043077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92c410-ae09-4ee0-965e-22a850683f00 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:04.053415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1089/2172] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:04.082667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813f784-e085-46ad-8486-496c3b141114 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:04.154574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813f784-e085-46ad-8486-496c3b141114 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:04.159444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:04.160383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:04.160662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:04.160939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] PATH_INFO: `/v3/users/73d18de5ef504b1dbf78793ed678006c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:04.162215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d18de5ef504b1dbf78793ed678006c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:04.164245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:04.220252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d18de5ef504b1dbf78793ed678006c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.235380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d18de5ef504b1dbf78793ed678006c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.244770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d18de5ef504b1dbf78793ed678006c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.252838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d18de5ef504b1dbf78793ed678006c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.253579 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d18de5ef504b1dbf78793ed678006c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.260223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1084/2173] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:56:04 2022] DELETE /identity/v3/users/73d18de5ef504b1dbf78793ed678006c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:04.264856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0882416-fc1d-4130-9acf-badc9c3d2905 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:04.265142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0882416-fc1d-4130-9acf-badc9c3d2905 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:04.265422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0882416-fc1d-4130-9acf-badc9c3d2905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:04.283585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f0882416-fc1d-4130-9acf-badc9c3d2905 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:04.284826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f0882416-fc1d-4130-9acf-badc9c3d2905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:04.347837 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1090/2174] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:04.358336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:04.360153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:04.382763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:04.429366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1630fb4-4689-4fde-8c87-408af01d8570 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:04.434451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:04.435251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:04.435646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:04.435927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:04.446749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:04.448197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:04.457903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1085/2175] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:04.738492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0882416-fc1d-4130-9acf-badc9c3d2905 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:04.740015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f0882416-fc1d-4130-9acf-badc9c3d2905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:04.753149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0882416-fc1d-4130-9acf-badc9c3d2905 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:04.758628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:04.759515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:04.759796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:04.760075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] PATH_INFO: `/v3/projects/4ed83119a201412e9edc9b3b10cd2a03` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:04.761406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed83119a201412e9edc9b3b10cd2a03)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:04.763442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:04.815326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed83119a201412e9edc9b3b10cd2a03', 'request_id': 'req-c4ea2189-f3fc-433e-8663-a9bead7533c6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.823299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed83119a201412e9edc9b3b10cd2a03', 'request_id': 'req-c4ea2189-f3fc-433e-8663-a9bead7533c6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.829992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed83119a201412e9edc9b3b10cd2a03', 'request_id': 'req-c4ea2189-f3fc-433e-8663-a9bead7533c6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.844045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] The token cache is being invalidate because project 4ed83119a201412e9edc9b3b10cd2a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:56:04.844820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.845137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c4ea2189-f3fc-433e-8663-a9bead7533c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:04.846697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1091/2176] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:56:04 2022] DELETE /identity/v3/projects/4ed83119a201412e9edc9b3b10cd2a03 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:13.428209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:13.431025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:13.444837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b1253-dcc4-4793-bace-46b089c4b9cd service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:13.520458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:13.521684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:13.522118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:13.522395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] PATH_INFO: `/v3/users/65b2c230e0a6451e88c3bced68ca3892` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:13.523943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b2c230e0a6451e88c3bced68ca3892)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:13.526111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:13.581336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b2c230e0a6451e88c3bced68ca3892'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:13.594766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b2c230e0a6451e88c3bced68ca3892'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:13.604655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b2c230e0a6451e88c3bced68ca3892'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:13.613472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b2c230e0a6451e88c3bced68ca3892'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:13.614349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b2c230e0a6451e88c3bced68ca3892'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:13.622422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1086/2177] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:56:13 2022] DELETE /identity/v3/users/65b2c230e0a6451e88c3bced68ca3892 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:56:13.627517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ccd9-9fb8-427f-b2cd-0c9976094282 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:13.628152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ccd9-9fb8-427f-b2cd-0c9976094282 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:13.628152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4ccd9-9fb8-427f-b2cd-0c9976094282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:13.650154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2be4ccd9-9fb8-427f-b2cd-0c9976094282 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:13.651475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2be4ccd9-9fb8-427f-b2cd-0c9976094282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:13.700690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:13.725163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1092/2178] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:56:13.735396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:13.737040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:13.789304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:13.809632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6f6d51-1602-432b-b829-b322f331f35e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:13.814774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f69c8-f9b7-418f-80f0-acc3735c36f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:13.815557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f69c8-f9b7-418f-80f0-acc3735c36f2 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:13.815892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f69c8-f9b7-418f-80f0-acc3735c36f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:13.816130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f69c8-f9b7-418f-80f0-acc3735c36f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:13.828778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f69c8-f9b7-418f-80f0-acc3735c36f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:13.831282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f69c8-f9b7-418f-80f0-acc3735c36f2 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:13.843129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1087/2179] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:14.161254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4ccd9-9fb8-427f-b2cd-0c9976094282 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:14.162869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2be4ccd9-9fb8-427f-b2cd-0c9976094282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:14.175224 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4ccd9-9fb8-427f-b2cd-0c9976094282 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:14.182997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:14.184996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.185366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.186115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] PATH_INFO: `/v3/projects/0a96ea38c3b64eddb3458c4a24b1f2e2` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.187498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a96ea38c3b64eddb3458c4a24b1f2e2)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:14.189435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:14.261997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a96ea38c3b64eddb3458c4a24b1f2e2', 'request_id': 'req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:14.272314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a96ea38c3b64eddb3458c4a24b1f2e2', 'request_id': 'req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:14.283101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a96ea38c3b64eddb3458c4a24b1f2e2', 'request_id': 'req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:14.295250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] The token cache is being invalidate because project 0a96ea38c3b64eddb3458c4a24b1f2e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:56:14.295559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:14.296252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-733b457c-9a2c-40c2-90b1-0eb93b29d8aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:14.297968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1093/2180] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:56:14 2022] DELETE /identity/v3/projects/0a96ea38c3b64eddb3458c4a24b1f2e2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:14.303259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d5e2-ff29-4811-8322-f488bf56e77d None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.303547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d5e2-ff29-4811-8322-f488bf56e77d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.303855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347d5e2-ff29-4811-8322-f488bf56e77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.335441 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b347d5e2-ff29-4811-8322-f488bf56e77d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:14.336941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b347d5e2-ff29-4811-8322-f488bf56e77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:14.406089 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1088/2181] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:56:14.414537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a646-2c85-49e5-a728-e09e43a98140 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.414537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a646-2c85-49e5-a728-e09e43a98140 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.414799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a646-2c85-49e5-a728-e09e43a98140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.439406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-bba4a646-2c85-49e5-a728-e09e43a98140 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:14.440666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bba4a646-2c85-49e5-a728-e09e43a98140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:14.451139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1094/2182] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:14.456978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347d5e2-ff29-4811-8322-f488bf56e77d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:14.468183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b347d5e2-ff29-4811-8322-f488bf56e77d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:14.474881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:14.475785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.476013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.476319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.477125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:56:14.477479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:14.479374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:14.500009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1089/2183] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:56:14.504133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4a646-2c85-49e5-a728-e09e43a98140 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:14.514827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4a646-2c85-49e5-a728-e09e43a98140 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:14.519916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0601-d4dd-405d-9cc7-6f41fd0cb649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:14.520590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0601-d4dd-405d-9cc7-6f41fd0cb649 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.520854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0601-d4dd-405d-9cc7-6f41fd0cb649 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.521281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0601-d4dd-405d-9cc7-6f41fd0cb649 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.522033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f0601-d4dd-405d-9cc7-6f41fd0cb649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:14.524699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f0601-d4dd-405d-9cc7-6f41fd0cb649 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:14.573821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1095/2184] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:56:14 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:14.577118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:14.595212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8c4f2-b3f2-4aa2-8047-377e2362e2c3 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:14.669966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e0c7a-e2c5-4d76-83d2-2b3b20f6207c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:14.670654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e0c7a-e2c5-4d76-83d2-2b3b20f6207c admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.670928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e0c7a-e2c5-4d76-83d2-2b3b20f6207c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.671205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e0c7a-e2c5-4d76-83d2-2b3b20f6207c admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.671926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e0c7a-e2c5-4d76-83d2-2b3b20f6207c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:14.674000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e0c7a-e2c5-4d76-83d2-2b3b20f6207c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:14.698995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1090/2185] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:56:14 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:56:14.706981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0601-d4dd-405d-9cc7-6f41fd0cb649 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:14.722115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0601-d4dd-405d-9cc7-6f41fd0cb649 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:14.730380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af5eb7-eb24-4155-ba4f-1e0ee2d1dded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:14.731268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5eb7-eb24-4155-ba4f-1e0ee2d1dded admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.731553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5eb7-eb24-4155-ba4f-1e0ee2d1dded admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.731996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af5eb7-eb24-4155-ba4f-1e0ee2d1dded admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.732571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af5eb7-eb24-4155-ba4f-1e0ee2d1dded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:14.734471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af5eb7-eb24-4155-ba4f-1e0ee2d1dded admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:14.749453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1096/2186] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:14.750671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e0c7a-e2c5-4d76-83d2-2b3b20f6207c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:14.761982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e0c7a-e2c5-4d76-83d2-2b3b20f6207c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:14.767186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda11cb-54e1-4a2b-ad8c-e142685d7217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:14.767942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda11cb-54e1-4a2b-ad8c-e142685d7217 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.768316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda11cb-54e1-4a2b-ad8c-e142685d7217 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.768676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda11cb-54e1-4a2b-ad8c-e142685d7217 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.769498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda11cb-54e1-4a2b-ad8c-e142685d7217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:14.771240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda11cb-54e1-4a2b-ad8c-e142685d7217 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:14.782573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1091/2187] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:14.791094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af5eb7-eb24-4155-ba4f-1e0ee2d1dded admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:14.803816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af5eb7-eb24-4155-ba4f-1e0ee2d1dded admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:14.809793 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:14.810792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.811143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.811557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] PATH_INFO: `/v3/projects/661449a8a10e4d14afad24c30d845db5/users/a45070e6409f460196d7560b2eb4becd/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.828661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] RBAC: Authorizing `identity:create_grant(project_id=661449a8a10e4d14afad24c30d845db5, user_id=a45070e6409f460196d7560b2eb4becd, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:14.831643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:14.847613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1097/2188] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:56:14 2022] PUT /identity/v3/projects/661449a8a10e4d14afad24c30d845db5/users/a45070e6409f460196d7560b2eb4becd/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:14.859458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992ec6a-b645-44be-8b8b-02255e18d4af None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:14.859788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992ec6a-b645-44be-8b8b-02255e18d4af None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:14.860065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992ec6a-b645-44be-8b8b-02255e18d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:14.890385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-7992ec6a-b645-44be-8b8b-02255e18d4af None None] MFA Rules not processed for user `a45070e6409f460196d7560b2eb4becd`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:14.891642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-7992ec6a-b645-44be-8b8b-02255e18d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:14.921065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:14.963871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1092/2189] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:14.976096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:14.978662 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:15.053275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc333943-1991-43af-b541-617fd9b5c73c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:15.058825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:15.060029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:15.060322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:15.060597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:15.071416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:15.072875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:15.084170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1098/2190] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:16.478780 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2b103-18c9-4e0a-a65b-9ffae0de0c6e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:16.479308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2b103-18c9-4e0a-a65b-9ffae0de0c6e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:16.479308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2b103-18c9-4e0a-a65b-9ffae0de0c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:16.499090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-9fe2b103-18c9-4e0a-a65b-9ffae0de0c6e None None] MFA Rules not processed for user `a45070e6409f460196d7560b2eb4becd`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:16.500548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9fe2b103-18c9-4e0a-a65b-9ffae0de0c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:16.521187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1093/2191] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:56:17.984906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:17.986580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:17.995200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:18.082434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661f6da-b800-4a51-83fc-f95930397170 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:18.087632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:18.088517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:18.088799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:18.089079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] PATH_INFO: `/v3/users/d7c5e4602e9f4845a1a498b702c567d5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:18.090394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7c5e4602e9f4845a1a498b702c567d5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:18.092338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:18.153159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7c5e4602e9f4845a1a498b702c567d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.169921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7c5e4602e9f4845a1a498b702c567d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.179910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7c5e4602e9f4845a1a498b702c567d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.180914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7c5e4602e9f4845a1a498b702c567d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.194119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7c5e4602e9f4845a1a498b702c567d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.203139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1099/2192] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:56:17 2022] DELETE /identity/v3/users/d7c5e4602e9f4845a1a498b702c567d5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:18.207866 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8400bfb-4560-4857-9d31-4929f78afb39 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:18.208584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8400bfb-4560-4857-9d31-4929f78afb39 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:18.208584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8400bfb-4560-4857-9d31-4929f78afb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:18.230342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a8400bfb-4560-4857-9d31-4929f78afb39 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:18.231741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a8400bfb-4560-4857-9d31-4929f78afb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:18.299401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1094/2193] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:56:18.312363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:18.313907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:18.380978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3f41d-2cee-4703-897f-f4dc1cc89146 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:18.386996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ac4b4-31bc-4f6c-adc8-bbdea995757d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:18.388048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac4b4-31bc-4f6c-adc8-bbdea995757d service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:18.388466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac4b4-31bc-4f6c-adc8-bbdea995757d service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:18.388867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac4b4-31bc-4f6c-adc8-bbdea995757d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:18.399354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ac4b4-31bc-4f6c-adc8-bbdea995757d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:18.400957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ac4b4-31bc-4f6c-adc8-bbdea995757d service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:18.411409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1100/2194] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:18.734491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8400bfb-4560-4857-9d31-4929f78afb39 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:18.735965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a8400bfb-4560-4857-9d31-4929f78afb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:18.748859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8400bfb-4560-4857-9d31-4929f78afb39 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:18.754806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:18.755694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:18.755968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:18.756597 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] PATH_INFO: `/v3/projects/06db592518d94f498a57c52121e32ca0` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:18.758643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06db592518d94f498a57c52121e32ca0)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:18.760563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:18.812725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06db592518d94f498a57c52121e32ca0', 'request_id': 'req-8d70d831-5979-4400-b843-790564f3bc53'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.820225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06db592518d94f498a57c52121e32ca0', 'request_id': 'req-8d70d831-5979-4400-b843-790564f3bc53'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.828969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06db592518d94f498a57c52121e32ca0', 'request_id': 'req-8d70d831-5979-4400-b843-790564f3bc53'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.841220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] The token cache is being invalidate because project 06db592518d94f498a57c52121e32ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:56:18.841524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.842265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8d70d831-5979-4400-b843-790564f3bc53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:18.843865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1095/2195] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:56:18 2022] DELETE /identity/v3/projects/06db592518d94f498a57c52121e32ca0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:18.849423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2511e-d086-4547-a919-d98f8bf5240c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:18.849820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2511e-d086-4547-a919-d98f8bf5240c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:18.850205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2511e-d086-4547-a919-d98f8bf5240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:18.871080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3eb2511e-d086-4547-a919-d98f8bf5240c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:18.872279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3eb2511e-d086-4547-a919-d98f8bf5240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:18.895266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:18.942578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1101/2196] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:18.951224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2f0ac-9c76-49ab-b4d7-c98e058a74dd None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:18.951600 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2f0ac-9c76-49ab-b4d7-c98e058a74dd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:18.951884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2f0ac-9c76-49ab-b4d7-c98e058a74dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:18.976536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-47a2f0ac-9c76-49ab-b4d7-c98e058a74dd None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:18.978102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-47a2f0ac-9c76-49ab-b4d7-c98e058a74dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:18.988931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1096/2197] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:18.993486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2511e-d086-4547-a919-d98f8bf5240c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:19.004729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2511e-d086-4547-a919-d98f8bf5240c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:19.011860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:19.013360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:19.013651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:19.013966 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:19.014728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:56:19.015580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:19.017476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:19.032262 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1102/2198] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:56:19.037007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2f0ac-9c76-49ab-b4d7-c98e058a74dd None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:19.047842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2f0ac-9c76-49ab-b4d7-c98e058a74dd None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:19.054458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab4f2-7cc0-4639-8cd8-3419057db2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:19.055316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab4f2-7cc0-4639-8cd8-3419057db2aa admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:19.055598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab4f2-7cc0-4639-8cd8-3419057db2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:19.055901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ab4f2-7cc0-4639-8cd8-3419057db2aa admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:19.056665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ab4f2-7cc0-4639-8cd8-3419057db2aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:19.058986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ab4f2-7cc0-4639-8cd8-3419057db2aa admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:19.101207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1097/2199] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:56:19 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:56:19.106723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:19.120267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4f7df-c567-40f6-8ebc-121aad04a790 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:19.207364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b112b-edf1-4123-8b9a-fa9c78fecdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:19.208185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b112b-edf1-4123-8b9a-fa9c78fecdef admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:19.208474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b112b-edf1-4123-8b9a-fa9c78fecdef admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:19.208752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b112b-edf1-4123-8b9a-fa9c78fecdef admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:19.209617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b112b-edf1-4123-8b9a-fa9c78fecdef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:19.212027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b112b-edf1-4123-8b9a-fa9c78fecdef admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:19.250001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1103/2200] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:56:19 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:19.251916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab4f2-7cc0-4639-8cd8-3419057db2aa admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:19.263557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ab4f2-7cc0-4639-8cd8-3419057db2aa admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:19.268846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e94ca1-e89a-49dd-bca9-8c85972cf3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:19.269847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e94ca1-e89a-49dd-bca9-8c85972cf3cb admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:19.270351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e94ca1-e89a-49dd-bca9-8c85972cf3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:19.270810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e94ca1-e89a-49dd-bca9-8c85972cf3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:19.271896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e94ca1-e89a-49dd-bca9-8c85972cf3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:19.274075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e94ca1-e89a-49dd-bca9-8c85972cf3cb admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:19.289971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1098/2201] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:19.293895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b112b-edf1-4123-8b9a-fa9c78fecdef admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:19.306969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b112b-edf1-4123-8b9a-fa9c78fecdef admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:19.311924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77cc9a-4ef5-47df-bd8d-66d2e0be774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:19.312682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cc9a-4ef5-47df-bd8d-66d2e0be774b admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:19.313494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cc9a-4ef5-47df-bd8d-66d2e0be774b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:19.313871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77cc9a-4ef5-47df-bd8d-66d2e0be774b admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:19.314705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77cc9a-4ef5-47df-bd8d-66d2e0be774b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:19.316389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77cc9a-4ef5-47df-bd8d-66d2e0be774b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:19.329525 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1104/2202] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:19.333992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e94ca1-e89a-49dd-bca9-8c85972cf3cb admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:19.356794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e94ca1-e89a-49dd-bca9-8c85972cf3cb admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:19.362907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:19.364179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:19.364488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:19.364767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] PATH_INFO: `/v3/projects/5ec57874f76e48f9ac7e578f5176d063/users/3ded85fa90c54120825310cce7635286/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:19.367774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:19.384289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec57874f76e48f9ac7e578f5176d063, user_id=3ded85fa90c54120825310cce7635286, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:19.387217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:19.405137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1099/2203] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:56:19 2022] PUT /identity/v3/projects/5ec57874f76e48f9ac7e578f5176d063/users/3ded85fa90c54120825310cce7635286/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:19.416090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e160-d1a6-4fe9-9a55-7f9d56e68f50 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:19.416386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e160-d1a6-4fe9-9a55-7f9d56e68f50 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:19.416668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e160-d1a6-4fe9-9a55-7f9d56e68f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:19.450831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-aba6e160-d1a6-4fe9-9a55-7f9d56e68f50 None None] MFA Rules not processed for user `3ded85fa90c54120825310cce7635286`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:19.452079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-aba6e160-d1a6-4fe9-9a55-7f9d56e68f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:19.531066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1105/2204] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:19.544439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:19.546442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:19.617797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec7084-7737-46a9-8f1f-02079dd51dab admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:19.619192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160cb78-8732-45d0-bd1c-e7c7ba22dc1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:19.619916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160cb78-8732-45d0-bd1c-e7c7ba22dc1a service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:19.620196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160cb78-8732-45d0-bd1c-e7c7ba22dc1a service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:19.620527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160cb78-8732-45d0-bd1c-e7c7ba22dc1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:19.630591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160cb78-8732-45d0-bd1c-e7c7ba22dc1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:19.632923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160cb78-8732-45d0-bd1c-e7c7ba22dc1a service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:19.643400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1100/2205] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:29.527209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:29.638547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:29.743676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6e160-d1a6-4fe9-9a55-7f9d56e68f50 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:29.813719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6e160-d1a6-4fe9-9a55-7f9d56e68f50 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:29.817568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:29.818489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:29.818957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:29.819764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] PATH_INFO: `/v3/users/3ded85fa90c54120825310cce7635286` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:29.820958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ded85fa90c54120825310cce7635286)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:29.823071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:29.883249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ded85fa90c54120825310cce7635286'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:29.895202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ded85fa90c54120825310cce7635286'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:29.904077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ded85fa90c54120825310cce7635286'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:29.904708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ded85fa90c54120825310cce7635286'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:29.911574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ded85fa90c54120825310cce7635286'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:29.918601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1106/2206] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:56:29 2022] DELETE /identity/v3/users/3ded85fa90c54120825310cce7635286 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:29.923198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3f955-ae1b-42ba-8ffc-6d3ba244dd2d None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:29.923485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3f955-ae1b-42ba-8ffc-6d3ba244dd2d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:29.923759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3f955-ae1b-42ba-8ffc-6d3ba244dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:29.942716 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f9a3f955-ae1b-42ba-8ffc-6d3ba244dd2d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:29.944562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f9a3f955-ae1b-42ba-8ffc-6d3ba244dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:30.010460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1101/2207] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:30.022505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:30.024221 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:30.097075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f51ba-58b9-43e9-a642-41a33c1ac845 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:30.104395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33009a1a-3bc0-477e-8b08-0f237d188e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:30.105344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33009a1a-3bc0-477e-8b08-0f237d188e88 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.105705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33009a1a-3bc0-477e-8b08-0f237d188e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.106131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33009a1a-3bc0-477e-8b08-0f237d188e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.124674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33009a1a-3bc0-477e-8b08-0f237d188e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:30.128350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33009a1a-3bc0-477e-8b08-0f237d188e88 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:30.145057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1107/2208] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:30.432428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3f955-ae1b-42ba-8ffc-6d3ba244dd2d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:30.434214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f9a3f955-ae1b-42ba-8ffc-6d3ba244dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:30.448606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3f955-ae1b-42ba-8ffc-6d3ba244dd2d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:30.453931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:30.454848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.455143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.455417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] PATH_INFO: `/v3/projects/5ec57874f76e48f9ac7e578f5176d063` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.457760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec57874f76e48f9ac7e578f5176d063)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:30.458730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:30.526207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec57874f76e48f9ac7e578f5176d063', 'request_id': 'req-0a7541d9-28e3-471c-81b1-8253b2aad224'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:30.536739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec57874f76e48f9ac7e578f5176d063', 'request_id': 'req-0a7541d9-28e3-471c-81b1-8253b2aad224'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:30.554474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec57874f76e48f9ac7e578f5176d063', 'request_id': 'req-0a7541d9-28e3-471c-81b1-8253b2aad224'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:30.572353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] The token cache is being invalidate because project 5ec57874f76e48f9ac7e578f5176d063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:56:30.572666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:30.573490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a7541d9-28e3-471c-81b1-8253b2aad224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:30.575166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1102/2209] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:56:30 2022] DELETE /identity/v3/projects/5ec57874f76e48f9ac7e578f5176d063 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:30.580299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b296b-8341-4fff-8318-b6494b61e3b1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.580805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b296b-8341-4fff-8318-b6494b61e3b1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.580912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b296b-8341-4fff-8318-b6494b61e3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.601466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-129b296b-8341-4fff-8318-b6494b61e3b1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:30.602809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-129b296b-8341-4fff-8318-b6494b61e3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:30.622720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:30.672510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1108/2210] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:30.681721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e11b-103c-4f24-b6cf-e2892a666a97 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.682372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e11b-103c-4f24-b6cf-e2892a666a97 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.682372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e11b-103c-4f24-b6cf-e2892a666a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.703404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b157e11b-103c-4f24-b6cf-e2892a666a97 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:30.704790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b157e11b-103c-4f24-b6cf-e2892a666a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:30.714793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1103/2211] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:30.722874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b296b-8341-4fff-8318-b6494b61e3b1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:30.733798 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b296b-8341-4fff-8318-b6494b61e3b1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:30.738853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:30.739552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.739830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.740098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.740988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:56:30.741305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:30.742962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:30.755663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1109/2212] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:56:30.760712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157e11b-103c-4f24-b6cf-e2892a666a97 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:30.770626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157e11b-103c-4f24-b6cf-e2892a666a97 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:30.776526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be9ed9-3b81-466d-b58b-4e46b99bd392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:30.777215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be9ed9-3b81-466d-b58b-4e46b99bd392 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.777495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be9ed9-3b81-466d-b58b-4e46b99bd392 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.777752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be9ed9-3b81-466d-b58b-4e46b99bd392 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.778494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be9ed9-3b81-466d-b58b-4e46b99bd392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:30.780447 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be9ed9-3b81-466d-b58b-4e46b99bd392 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:30.808883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1104/2213] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:56:30 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:30.813826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:30.825381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa914453-1718-4adc-aa43-8c1c6bd47d18 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:30.895407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f807d0-a65e-4167-9fbd-94ae7b90bfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:30.896025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f807d0-a65e-4167-9fbd-94ae7b90bfc3 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.896290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f807d0-a65e-4167-9fbd-94ae7b90bfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.896565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f807d0-a65e-4167-9fbd-94ae7b90bfc3 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.897714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f807d0-a65e-4167-9fbd-94ae7b90bfc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:30.899905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f807d0-a65e-4167-9fbd-94ae7b90bfc3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:30.924077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1110/2214] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:56:30 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:30.928932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be9ed9-3b81-466d-b58b-4e46b99bd392 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:30.940366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be9ed9-3b81-466d-b58b-4e46b99bd392 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:30.945923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbbd36-288e-4781-a3de-e33286cd68ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:30.946924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbbd36-288e-4781-a3de-e33286cd68ff admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.947297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbbd36-288e-4781-a3de-e33286cd68ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.947612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbbd36-288e-4781-a3de-e33286cd68ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.948387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbbd36-288e-4781-a3de-e33286cd68ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:30.950048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbbd36-288e-4781-a3de-e33286cd68ff admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:30.964649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1105/2215] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:30.966059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f807d0-a65e-4167-9fbd-94ae7b90bfc3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:30.981302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f807d0-a65e-4167-9fbd-94ae7b90bfc3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:30.986258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625533bb-6c22-4172-83ad-f3fc621dac1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:30.986961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625533bb-6c22-4172-83ad-f3fc621dac1e admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:30.987243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625533bb-6c22-4172-83ad-f3fc621dac1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:30.987520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625533bb-6c22-4172-83ad-f3fc621dac1e admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:30.988242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625533bb-6c22-4172-83ad-f3fc621dac1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:30.989923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625533bb-6c22-4172-83ad-f3fc621dac1e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:31.003709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1111/2216] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 15:56:31.005261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbbd36-288e-4781-a3de-e33286cd68ff admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:31.015955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbbd36-288e-4781-a3de-e33286cd68ff admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:31.020750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:31.022146 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:31.022288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:31.022489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] PATH_INFO: `/v3/projects/af3eba9c8dda4166accce40c556af46e/users/32512d9ebcc8447a88b99da4181895e7/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:31.040313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3eba9c8dda4166accce40c556af46e, user_id=32512d9ebcc8447a88b99da4181895e7, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:31.043132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:31.058048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1106/2217] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:56:31 2022] PUT /identity/v3/projects/af3eba9c8dda4166accce40c556af46e/users/32512d9ebcc8447a88b99da4181895e7/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:31.063334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d45fa5-9c07-4285-ba72-4b4a6fcc2371 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:31.063869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d45fa5-9c07-4285-ba72-4b4a6fcc2371 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:31.064169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d45fa5-9c07-4285-ba72-4b4a6fcc2371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:31.088754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-84d45fa5-9c07-4285-ba72-4b4a6fcc2371 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:31.090062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-84d45fa5-9c07-4285-ba72-4b4a6fcc2371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:31.159373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1112/2218] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:56:31.170217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:31.172062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:31.197218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:31.238279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78571bd3-d065-4799-b7cc-49885932f9b3 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:31.243210 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:31.244216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:31.244504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:31.244783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:31.261872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:31.264456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:31.277294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1107/2219] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:32.204128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f140a34-d3ea-4dac-a919-3ff9ec113a77 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:32.204660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f140a34-d3ea-4dac-a919-3ff9ec113a77 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:32.204660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f140a34-d3ea-4dac-a919-3ff9ec113a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:32.265183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1f140a34-d3ea-4dac-a919-3ff9ec113a77 None None] MFA Rules not processed for user `32512d9ebcc8447a88b99da4181895e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:32.267936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1f140a34-d3ea-4dac-a919-3ff9ec113a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:32.289127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:32.348275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1113/2220] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:32.359197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:32.360594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:32.443442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4579e-24fd-46c2-985c-1e107290360f service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:32.447506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:32.448164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:32.448451 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:32.448726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:32.458375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:32.459921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:32.468977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1108/2221] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:33.002088 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400b92-deeb-4d3d-a5b8-fd845ade13aa None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:33.002378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400b92-deeb-4d3d-a5b8-fd845ade13aa None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:33.002655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e400b92-deeb-4d3d-a5b8-fd845ade13aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:33.032840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2e400b92-deeb-4d3d-a5b8-fd845ade13aa None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:33.034120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2e400b92-deeb-4d3d-a5b8-fd845ade13aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:33.107469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1114/2222] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:33.148585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:33.150781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:33.228171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e0392-9ee3-4f9a-b877-a49f95aec168 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:33.234708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620aae10-af12-4611-ae97-26da95f8900d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:33.235378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aae10-af12-4611-ae97-26da95f8900d service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:33.235657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aae10-af12-4611-ae97-26da95f8900d service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:33.235937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aae10-af12-4611-ae97-26da95f8900d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:33.250669 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620aae10-af12-4611-ae97-26da95f8900d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:33.253068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620aae10-af12-4611-ae97-26da95f8900d service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:33.262306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1109/2223] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:56:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:56:33.832276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6367-3937-4a44-8990-e4f1e0ce06aa None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:33.832776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6367-3937-4a44-8990-e4f1e0ce06aa None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:33.832873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e6367-3937-4a44-8990-e4f1e0ce06aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:33.835712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc597dc-830d-4111-a732-de78d51912fc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:33.836162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc597dc-830d-4111-a732-de78d51912fc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:33.836292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc597dc-830d-4111-a732-de78d51912fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:33.851955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-a88e6367-3937-4a44-8990-e4f1e0ce06aa None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:33.853159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a88e6367-3937-4a44-8990-e4f1e0ce06aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:33.854823 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-6bc597dc-830d-4111-a732-de78d51912fc None None] MFA Rules not processed for user `32512d9ebcc8447a88b99da4181895e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:33.855989 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6bc597dc-830d-4111-a732-de78d51912fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:33.860447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1115/2224] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jun 21 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:33.863582 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1110/2225] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:33.873236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e6367-3937-4a44-8990-e4f1e0ce06aa None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:33.885957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e6367-3937-4a44-8990-e4f1e0ce06aa None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:33.891227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96716ab-0c6f-486c-9bae-a877c313dd72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:33.891879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96716ab-0c6f-486c-9bae-a877c313dd72 service swift] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:33.892225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96716ab-0c6f-486c-9bae-a877c313dd72 service swift] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:33.892418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96716ab-0c6f-486c-9bae-a877c313dd72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:33.902049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96716ab-0c6f-486c-9bae-a877c313dd72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:33.903647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96716ab-0c6f-486c-9bae-a877c313dd72 service swift] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:33.913442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1116/2226] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:56:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:56:34.007269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc597dc-830d-4111-a732-de78d51912fc None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:34.009426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6bc597dc-830d-4111-a732-de78d51912fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:34.051131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:34.130593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc597dc-830d-4111-a732-de78d51912fc None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:34.139828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:34.140609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:34.141247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:34.141577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:34.143623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:34.164276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:34.167078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:34.182375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195f56-74d1-4940-aa82-0c313adc0c89 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:34.182748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195f56-74d1-4940-aa82-0c313adc0c89 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:34.182871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1111/2227] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:34.183363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72195f56-74d1-4940-aa82-0c313adc0c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:34.202829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-72195f56-74d1-4940-aa82-0c313adc0c89 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:34.204046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-72195f56-74d1-4940-aa82-0c313adc0c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:34.212241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1117/2228] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:56:34.225922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:34.241609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba89a5-e054-4e42-a0b8-0a9e27551a8f service placement] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:34.253659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165bd18-f5e5-45fd-ac9f-5f32689461bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:34.254506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165bd18-f5e5-45fd-ac9f-5f32689461bc service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:34.254797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165bd18-f5e5-45fd-ac9f-5f32689461bc service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:34.257263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165bd18-f5e5-45fd-ac9f-5f32689461bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:34.268428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165bd18-f5e5-45fd-ac9f-5f32689461bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:34.270444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165bd18-f5e5-45fd-ac9f-5f32689461bc service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:34.280624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1112/2229] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:56:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:56:34.691129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26a7c5-d78d-48b9-88be-42c16329afce None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:34.691421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26a7c5-d78d-48b9-88be-42c16329afce None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:34.691734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26a7c5-d78d-48b9-88be-42c16329afce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:34.712413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-cd26a7c5-d78d-48b9-88be-42c16329afce None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:34.713861 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cd26a7c5-d78d-48b9-88be-42c16329afce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:34.721803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1118/2230] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jun 21 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:34.734479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165bd18-f5e5-45fd-ac9f-5f32689461bc service swift] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:34.748016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165bd18-f5e5-45fd-ac9f-5f32689461bc service swift] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:34.753795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecc428-211e-4c45-9056-4dd2ee6582b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:34.754494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc428-211e-4c45-9056-4dd2ee6582b6 service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:34.754781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc428-211e-4c45-9056-4dd2ee6582b6 service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:34.755062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc428-211e-4c45-9056-4dd2ee6582b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:34.764592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecc428-211e-4c45-9056-4dd2ee6582b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:34.766250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecc428-211e-4c45-9056-4dd2ee6582b6 service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:34.774743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1113/2231] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:56:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 15:56:36.945559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26a7c5-d78d-48b9-88be-42c16329afce None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:36.947895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cd26a7c5-d78d-48b9-88be-42c16329afce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:36.962789 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd26a7c5-d78d-48b9-88be-42c16329afce None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:36.968463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:36.969438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:36.969916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:36.970274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] PATH_INFO: `/v3/users/a45070e6409f460196d7560b2eb4becd` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:36.971588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45070e6409f460196d7560b2eb4becd)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:36.973995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:37.031291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45070e6409f460196d7560b2eb4becd'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.044480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45070e6409f460196d7560b2eb4becd'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.048341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:37.053770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45070e6409f460196d7560b2eb4becd'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.054493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45070e6409f460196d7560b2eb4becd'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.064199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45070e6409f460196d7560b2eb4becd'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.071838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1119/2232] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 15:56:36 2022] DELETE /identity/v3/users/a45070e6409f460196d7560b2eb4becd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:56:37.076330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8889-c293-4867-a87a-b3489f68da12 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.076788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8889-c293-4867-a87a-b3489f68da12 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.076909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a8889-c293-4867-a87a-b3489f68da12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.103142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ca6a8889-c293-4867-a87a-b3489f68da12 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:37.104350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ca6a8889-c293-4867-a87a-b3489f68da12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:37.166035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1114/2233] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 15:56:37.175960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:37.177905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:37.240242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98053d9e-a74b-4abf-a8eb-acea1fd21d69 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:37.244850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9b053-926e-4614-b50f-45a3eb95e440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:37.245714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9b053-926e-4614-b50f-45a3eb95e440 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.245955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9b053-926e-4614-b50f-45a3eb95e440 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.246230 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9b053-926e-4614-b50f-45a3eb95e440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.257679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9b053-926e-4614-b50f-45a3eb95e440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:37.259336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9b053-926e-4614-b50f-45a3eb95e440 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:37.270201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1120/2234] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:37.465535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a8889-c293-4867-a87a-b3489f68da12 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:37.467186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ca6a8889-c293-4867-a87a-b3489f68da12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:37.481191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a8889-c293-4867-a87a-b3489f68da12 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:37.487848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:37.488883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.489268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.489679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] PATH_INFO: `/v3/projects/661449a8a10e4d14afad24c30d845db5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.491461 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=661449a8a10e4d14afad24c30d845db5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:37.494460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:37.547857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661449a8a10e4d14afad24c30d845db5', 'request_id': 'req-56a3c583-118d-428a-8005-08e1116ae9ad'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.555297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661449a8a10e4d14afad24c30d845db5', 'request_id': 'req-56a3c583-118d-428a-8005-08e1116ae9ad'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.564809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661449a8a10e4d14afad24c30d845db5', 'request_id': 'req-56a3c583-118d-428a-8005-08e1116ae9ad'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.569976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:37.577117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] The token cache is being invalidate because project 661449a8a10e4d14afad24c30d845db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 15:56:37.577402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.578567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-56a3c583-118d-428a-8005-08e1116ae9ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 15:56:37.580224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1115/2235] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 15:56:37 2022] DELETE /identity/v3/projects/661449a8a10e4d14afad24c30d845db5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:37.585492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4208-2b71-4fc4-9f33-1e39ac54b33b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.585796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4208-2b71-4fc4-9f33-1e39ac54b33b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.586092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4208-2b71-4fc4-9f33-1e39ac54b33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.604718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-578c4208-2b71-4fc4-9f33-1e39ac54b33b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:37.605979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-578c4208-2b71-4fc4-9f33-1e39ac54b33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:37.674503 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1121/2236] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:37.684348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299df44-5bfe-4115-b5da-af413a0a26c9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.684658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299df44-5bfe-4115-b5da-af413a0a26c9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.685067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299df44-5bfe-4115-b5da-af413a0a26c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.712208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-4299df44-5bfe-4115-b5da-af413a0a26c9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:37.713420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-4299df44-5bfe-4115-b5da-af413a0a26c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:37.722446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1116/2237] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:37.727603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4208-2b71-4fc4-9f33-1e39ac54b33b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:37.737672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4208-2b71-4fc4-9f33-1e39ac54b33b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:37.742600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:37.743252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.743537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.743807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.744536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 15:56:37.744841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:37.746588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:37.758948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1122/2238] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 15:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 15:56:37.763319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299df44-5bfe-4115-b5da-af413a0a26c9 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:37.772905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299df44-5bfe-4115-b5da-af413a0a26c9 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:37.778340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c8545-7fbe-4d0b-ade4-2b418f19ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:37.778975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c8545-7fbe-4d0b-ade4-2b418f19ab52 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.779254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c8545-7fbe-4d0b-ade4-2b418f19ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.779528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c8545-7fbe-4d0b-ade4-2b418f19ab52 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.780220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c8545-7fbe-4d0b-ade4-2b418f19ab52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:37.782286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c8545-7fbe-4d0b-ade4-2b418f19ab52 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:37.813244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1117/2239] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:56:37 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:37.819336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:37.832927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c810da9-3ceb-4ac0-9d31-f9de4822d971 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:37.894206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedea468-1345-4e84-b9b7-683bc166c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:37.894879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea468-1345-4e84-b9b7-683bc166c8ed admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.895158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea468-1345-4e84-b9b7-683bc166c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.895428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea468-1345-4e84-b9b7-683bc166c8ed admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.896130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedea468-1345-4e84-b9b7-683bc166c8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:37.898109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedea468-1345-4e84-b9b7-683bc166c8ed admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:37.923804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1123/2240] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:56:37 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:37.928043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c8545-7fbe-4d0b-ade4-2b418f19ab52 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:37.939203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c8545-7fbe-4d0b-ade4-2b418f19ab52 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:37.945101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48736c0e-0869-4322-a38f-f16098c8b9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:37.946093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48736c0e-0869-4322-a38f-f16098c8b9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.946214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48736c0e-0869-4322-a38f-f16098c8b9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.946456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48736c0e-0869-4322-a38f-f16098c8b9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.947189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48736c0e-0869-4322-a38f-f16098c8b9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:37.948797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48736c0e-0869-4322-a38f-f16098c8b9d4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:37.959539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1118/2241] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:37.963575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedea468-1345-4e84-b9b7-683bc166c8ed admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:37.975132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedea468-1345-4e84-b9b7-683bc166c8ed admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:37.980375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9f4e2-cb41-42d4-bb78-ec148da79773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:37.981072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9f4e2-cb41-42d4-bb78-ec148da79773 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:37.981339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9f4e2-cb41-42d4-bb78-ec148da79773 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:37.981756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9f4e2-cb41-42d4-bb78-ec148da79773 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:37.982534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9f4e2-cb41-42d4-bb78-ec148da79773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:37.984157 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9f4e2-cb41-42d4-bb78-ec148da79773 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:37.985608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:37.997817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1124/2242] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:38.001676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48736c0e-0869-4322-a38f-f16098c8b9d4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:38.014590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48736c0e-0869-4322-a38f-f16098c8b9d4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:38.019221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:38.020109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:38.020385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:38.020656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] PATH_INFO: `/v3/projects/8716150d618c402fbb51a2a042c5ea5b/users/cc3f66be61f54ea8bac135cf7b3531dc/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:38.037051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8716150d618c402fbb51a2a042c5ea5b, user_id=cc3f66be61f54ea8bac135cf7b3531dc, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:38.040008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:38.054822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1119/2243] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:56:38 2022] PUT /identity/v3/projects/8716150d618c402fbb51a2a042c5ea5b/users/cc3f66be61f54ea8bac135cf7b3531dc/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:56:38.060287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e018a2c-9a2c-425d-b9c4-a30e69960435 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:38.060587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e018a2c-9a2c-425d-b9c4-a30e69960435 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:38.060901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e018a2c-9a2c-425d-b9c4-a30e69960435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:38.090116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-8e018a2c-9a2c-425d-b9c4-a30e69960435 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:38.091318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8e018a2c-9a2c-425d-b9c4-a30e69960435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:38.155134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1125/2244] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:38.164856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:38.166674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:38.227487 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524492a-7f71-40db-8467-ec411eb68a85 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:38.231349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b9ebb-bae7-4e80-97c4-b4a9ca1e8ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:38.232002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b9ebb-bae7-4e80-97c4-b4a9ca1e8ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:38.232286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b9ebb-bae7-4e80-97c4-b4a9ca1e8ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:38.232554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b9ebb-bae7-4e80-97c4-b4a9ca1e8ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:38.243510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b9ebb-bae7-4e80-97c4-b4a9ca1e8ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:38.244968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b9ebb-bae7-4e80-97c4-b4a9ca1e8ae5 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:38.255462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1120/2245] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 15:56:42.691759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16dd05-4fc0-4857-be5c-53d86a27f617 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:42.692311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16dd05-4fc0-4857-be5c-53d86a27f617 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:42.692311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16dd05-4fc0-4857-be5c-53d86a27f617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:42.721831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-bb16dd05-4fc0-4857-be5c-53d86a27f617 None None] MFA Rules not processed for user `cc3f66be61f54ea8bac135cf7b3531dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:42.723465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bb16dd05-4fc0-4857-be5c-53d86a27f617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:42.816523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1126/2246] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:42.818614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b9ebb-bae7-4e80-97c4-b4a9ca1e8ae5 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:42.828821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b9ebb-bae7-4e80-97c4-b4a9ca1e8ae5 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:42.834350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e9d02-15dc-4d1e-9f5a-4221ee2f94ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:42.835030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e9d02-15dc-4d1e-9f5a-4221ee2f94ef admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:42.835315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e9d02-15dc-4d1e-9f5a-4221ee2f94ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:42.835592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e9d02-15dc-4d1e-9f5a-4221ee2f94ef admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:42.836421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e9d02-15dc-4d1e-9f5a-4221ee2f94ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:42.838350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e9d02-15dc-4d1e-9f5a-4221ee2f94ef admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:42.870724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1121/2247] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 15:56:42 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 15:56:42.874634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16dd05-4fc0-4857-be5c-53d86a27f617 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:42.903799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16dd05-4fc0-4857-be5c-53d86a27f617 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:43.001547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e91f9-4251-48cc-a5ee-cbb905339136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:43.002775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e91f9-4251-48cc-a5ee-cbb905339136 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:43.003195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e91f9-4251-48cc-a5ee-cbb905339136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:43.003518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e91f9-4251-48cc-a5ee-cbb905339136 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:43.004912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e91f9-4251-48cc-a5ee-cbb905339136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:43.025537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e91f9-4251-48cc-a5ee-cbb905339136 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:43.059544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1127/2248] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 15:56:42 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 15:56:43.063606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e9d02-15dc-4d1e-9f5a-4221ee2f94ef admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:43.078525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e9d02-15dc-4d1e-9f5a-4221ee2f94ef admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:43.100395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae84c32-121c-4ec9-b9cc-50ac9d12cb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:43.101240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae84c32-121c-4ec9-b9cc-50ac9d12cb36 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:43.101525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae84c32-121c-4ec9-b9cc-50ac9d12cb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:43.101847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae84c32-121c-4ec9-b9cc-50ac9d12cb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:43.102622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae84c32-121c-4ec9-b9cc-50ac9d12cb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:43.104981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae84c32-121c-4ec9-b9cc-50ac9d12cb36 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:43.135373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1122/2249] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:43.140638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e91f9-4251-48cc-a5ee-cbb905339136 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:43.152565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e91f9-4251-48cc-a5ee-cbb905339136 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:43.164140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890cf2-e1d9-456f-b9c9-93cbc1f1ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:43.165374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890cf2-e1d9-456f-b9c9-93cbc1f1ccdc admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:43.166005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890cf2-e1d9-456f-b9c9-93cbc1f1ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:43.166461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890cf2-e1d9-456f-b9c9-93cbc1f1ccdc admin admin] PATH_INFO: `/v3/projects/fceb0977c8424b5199d1193c8630f542/users/c77fc86204d9404b9e3375a0fd2830d9/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:43.181316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:43.185143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37890cf2-e1d9-456f-b9c9-93cbc1f1ccdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fceb0977c8424b5199d1193c8630f542, user_id=c77fc86204d9404b9e3375a0fd2830d9, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:43.188591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37890cf2-e1d9-456f-b9c9-93cbc1f1ccdc admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:43.207726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1128/2250] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:56:43 2022] PUT /identity/v3/projects/fceb0977c8424b5199d1193c8630f542/users/c77fc86204d9404b9e3375a0fd2830d9/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 15:56:43.211760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae84c32-121c-4ec9-b9cc-50ac9d12cb36 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:43.224304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae84c32-121c-4ec9-b9cc-50ac9d12cb36 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:43.302728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090bb7f-5ade-4707-bda6-a8fd467be015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:43.303869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090bb7f-5ade-4707-bda6-a8fd467be015 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:43.304257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090bb7f-5ade-4707-bda6-a8fd467be015 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:43.304632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090bb7f-5ade-4707-bda6-a8fd467be015 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:43.305521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090bb7f-5ade-4707-bda6-a8fd467be015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:43.307361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090bb7f-5ade-4707-bda6-a8fd467be015 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:43.310051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:43.322180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1123/2251] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 15:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 15:56:43.324515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890cf2-e1d9-456f-b9c9-93cbc1f1ccdc admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:43.336848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890cf2-e1d9-456f-b9c9-93cbc1f1ccdc admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:43.341676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:43.342608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:43.342909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:43.343185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] PATH_INFO: `/v3/projects/fceb0977c8424b5199d1193c8630f542/users/c77fc86204d9404b9e3375a0fd2830d9/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:43.345391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fceb0977c8424b5199d1193c8630f542, user_id=c77fc86204d9404b9e3375a0fd2830d9, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:43.348471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:43.360569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1129/2252] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 15:56:43 2022] PUT /identity/v3/projects/fceb0977c8424b5199d1193c8630f542/users/c77fc86204d9404b9e3375a0fd2830d9/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 15:56:48.752289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996baa9c-7eab-4caa-b7eb-789cf9e39b43 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:48.752814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996baa9c-7eab-4caa-b7eb-789cf9e39b43 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:48.752814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996baa9c-7eab-4caa-b7eb-789cf9e39b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:48.781442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-996baa9c-7eab-4caa-b7eb-789cf9e39b43 None None] MFA Rules not processed for user `c77fc86204d9404b9e3375a0fd2830d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:48.782875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-996baa9c-7eab-4caa-b7eb-789cf9e39b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:48.856391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1124/2253] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:48.871148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:48.873207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:48.980824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605a4fd-9b29-4840-9f6c-7a76c435e6c8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:48.985296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:48.986301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:48.986702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:48.987067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:49.069411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:49.070934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:49.080335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1130/2254] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:56:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:56:49.222302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36707ab-d11d-42fe-87d2-44af891dc0e7 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:49.222593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36707ab-d11d-42fe-87d2-44af891dc0e7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:49.222868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36707ab-d11d-42fe-87d2-44af891dc0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:49.288088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d36707ab-d11d-42fe-87d2-44af891dc0e7 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:56:49.289301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d36707ab-d11d-42fe-87d2-44af891dc0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:49.364994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1125/2255] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:56:49.377564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:56:49.379205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:56:49.464323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:49.481161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2c8be-a799-4ab2-980e-b09704d9c96b service glance] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:56:49.487220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430002c-8f15-4524-8e75-78fef7a68ca7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:56:49.487890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430002c-8f15-4524-8e75-78fef7a68ca7 service swift] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:56:49.488319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430002c-8f15-4524-8e75-78fef7a68ca7 service swift] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:56:49.488601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430002c-8f15-4524-8e75-78fef7a68ca7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:56:49.508664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430002c-8f15-4524-8e75-78fef7a68ca7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:56:49.511175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430002c-8f15-4524-8e75-78fef7a68ca7 service swift] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:56:49.521544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1131/2256] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:56:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:56:59.361606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=75 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:56:59.516705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=8 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:57:00.086408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2531-b4f5-481d-9cb0-702fb8f7ab4f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:57:00.086700 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2531-b4f5-481d-9cb0-702fb8f7ab4f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:57:00.086982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2531-b4f5-481d-9cb0-702fb8f7ab4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:57:00.106993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-556d2531-b4f5-481d-9cb0-702fb8f7ab4f None None] MFA Rules not processed for user `cc3f66be61f54ea8bac135cf7b3531dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:57:00.108253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-556d2531-b4f5-481d-9cb0-702fb8f7ab4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:57:00.118883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1126/2257] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:57:02.621802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5b12e-35a5-4a9b-9277-2637f45bcdd6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:57:02.622334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5b12e-35a5-4a9b-9277-2637f45bcdd6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:57:02.622334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5b12e-35a5-4a9b-9277-2637f45bcdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:57:02.641377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d6a5b12e-35a5-4a9b-9277-2637f45bcdd6 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:57:02.642778 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d6a5b12e-35a5-4a9b-9277-2637f45bcdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:57:02.651658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1132/2258] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:57:02.667161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d2531-b4f5-481d-9cb0-702fb8f7ab4f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:57:02.681031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d2531-b4f5-481d-9cb0-702fb8f7ab4f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:57:02.686455 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35573d5-cfea-429d-9d2f-75c5d2b865c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:57:02.687118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35573d5-cfea-429d-9d2f-75c5d2b865c5 service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:57:02.687399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35573d5-cfea-429d-9d2f-75c5d2b865c5 service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:57:02.687662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35573d5-cfea-429d-9d2f-75c5d2b865c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:57:02.697799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35573d5-cfea-429d-9d2f-75c5d2b865c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:57:02.699509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35573d5-cfea-429d-9d2f-75c5d2b865c5 service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:57:02.709430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1127/2259] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:57:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:57:12.635996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:57:12.705009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=9 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:57:56.118247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e897-15a1-4bc8-ab64-d00b796f6466 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:57:56.119498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e897-15a1-4bc8-ab64-d00b796f6466 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:57:56.120042 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e897-15a1-4bc8-ab64-d00b796f6466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:57:56.139968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ac35e897-15a1-4bc8-ab64-d00b796f6466 None None] MFA Rules not processed for user `01571ac0b01845f78192269cacb23644`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:57:56.141191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ac35e897-15a1-4bc8-ab64-d00b796f6466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:57:56.213304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1133/2260] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:57:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:57:56.774378 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98615f4-e201-45eb-962f-63e77b3e771c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:57:56.775231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98615f4-e201-45eb-962f-63e77b3e771c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:57:56.775510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98615f4-e201-45eb-962f-63e77b3e771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:57:56.794918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a98615f4-e201-45eb-962f-63e77b3e771c None None] MFA Rules not processed for user `9a67a5c4b7034d22b78331b907cc20bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:57:56.796138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a98615f4-e201-45eb-962f-63e77b3e771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:57:56.856146 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1128/2261] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:57:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:58:02.306764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9c2f3-eec0-455e-b680-5ec7fda041c8 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:02.307343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9c2f3-eec0-455e-b680-5ec7fda041c8 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:02.307343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9c2f3-eec0-455e-b680-5ec7fda041c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:02.327381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ecc9c2f3-eec0-455e-b680-5ec7fda041c8 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:58:02.328614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ecc9c2f3-eec0-455e-b680-5ec7fda041c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:02.336763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1134/2262] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:58:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:58:02.389697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98615f4-e201-45eb-962f-63e77b3e771c None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:02.404704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98615f4-e201-45eb-962f-63e77b3e771c None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:02.411784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c76e0-967b-478a-91cd-ee6b05cf7900 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:02.412517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c76e0-967b-478a-91cd-ee6b05cf7900 service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:02.412863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c76e0-967b-478a-91cd-ee6b05cf7900 service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:02.413202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c76e0-967b-478a-91cd-ee6b05cf7900 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:02.446664 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c76e0-967b-478a-91cd-ee6b05cf7900 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:02.449209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c76e0-967b-478a-91cd-ee6b05cf7900 service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:02.463803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1129/2263] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:58:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:58:12.320539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=28 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:58:12.461251 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=31 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:58:12.542224 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9c2f3-eec0-455e-b680-5ec7fda041c8 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:12.543800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ecc9c2f3-eec0-455e-b680-5ec7fda041c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:12.693041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9c2f3-eec0-455e-b680-5ec7fda041c8 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:12.699649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f30c2-fb98-4e5e-9bcf-1ae91932db7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:12.700625 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f30c2-fb98-4e5e-9bcf-1ae91932db7d service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:12.701128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f30c2-fb98-4e5e-9bcf-1ae91932db7d service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:12.702213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f30c2-fb98-4e5e-9bcf-1ae91932db7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:12.716566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f30c2-fb98-4e5e-9bcf-1ae91932db7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:12.719364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f30c2-fb98-4e5e-9bcf-1ae91932db7d service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:12.741226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1135/2264] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:58:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:58:22.731003 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=26 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:58:22.878392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfee9b-030c-4b96-a087-3bc061d603fe None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:22.879351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfee9b-030c-4b96-a087-3bc061d603fe None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:22.879852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfee9b-030c-4b96-a087-3bc061d603fe None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:22.881456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1130/2265] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:58:22.889293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd173c0-a329-4b83-8ba3-2af8ee982dce None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:22.889740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd173c0-a329-4b83-8ba3-2af8ee982dce None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:22.890273 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd173c0-a329-4b83-8ba3-2af8ee982dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:22.962813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3fd173c0-a329-4b83-8ba3-2af8ee982dce None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:58:22.964067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3fd173c0-a329-4b83-8ba3-2af8ee982dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:23.057584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1136/2266] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:58:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:58:23.079045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfee9b-030c-4b96-a087-3bc061d603fe None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:23.080837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-13dfee9b-030c-4b96-a087-3bc061d603fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:23.102066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfee9b-030c-4b96-a087-3bc061d603fe None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:23.108168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538e738-253c-475b-bc02-f3b7d2f70057 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:23.109313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e738-253c-475b-bc02-f3b7d2f70057 service glance] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:23.109820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e738-253c-475b-bc02-f3b7d2f70057 service glance] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:23.110286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538e738-253c-475b-bc02-f3b7d2f70057 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:23.124947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538e738-253c-475b-bc02-f3b7d2f70057 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:23.126634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538e738-253c-475b-bc02-f3b7d2f70057 service glance] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:23.140452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1131/2267] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:58:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:58:23.221898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf915b-5610-4701-89e4-81662b56d229 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:23.222319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf915b-5610-4701-89e4-81662b56d229 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:23.222681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf915b-5610-4701-89e4-81662b56d229 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:23.224184 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1137/2268] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:58:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:58:23.229698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18691c87-0635-4324-9f1d-d2a2188179d3 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:23.230261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18691c87-0635-4324-9f1d-d2a2188179d3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:23.230514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18691c87-0635-4324-9f1d-d2a2188179d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:23.252319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-18691c87-0635-4324-9f1d-d2a2188179d3 None None] MFA Rules not processed for user `6efa4bed025741d492158b5c6e45c7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:58:23.253567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-18691c87-0635-4324-9f1d-d2a2188179d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:23.261363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1132/2269] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:58:23.280719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf915b-5610-4701-89e4-81662b56d229 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:23.292232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf915b-5610-4701-89e4-81662b56d229 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:23.296340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fe791-4dfe-4063-9f6e-c9a867054905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:23.297229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe791-4dfe-4063-9f6e-c9a867054905 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:23.297485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe791-4dfe-4063-9f6e-c9a867054905 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:23.297759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fe791-4dfe-4063-9f6e-c9a867054905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:23.307318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fe791-4dfe-4063-9f6e-c9a867054905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:23.308137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fe791-4dfe-4063-9f6e-c9a867054905 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:23.316914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1138/2270] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:58:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:58:23.418757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079947cb-1ac1-483b-a865-4715fa30d92d None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:23.419151 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079947cb-1ac1-483b-a865-4715fa30d92d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:23.419652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079947cb-1ac1-483b-a865-4715fa30d92d None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:23.421131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1133/2271] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 15:58:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 15:58:23.427173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442111a9-39e0-4697-afc4-81aed8b4e381 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:23.427558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442111a9-39e0-4697-afc4-81aed8b4e381 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:23.427927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442111a9-39e0-4697-afc4-81aed8b4e381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:23.447052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-442111a9-39e0-4697-afc4-81aed8b4e381 None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:58:23.448287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-442111a9-39e0-4697-afc4-81aed8b4e381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:23.458446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1139/2272] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:58:23.470268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079947cb-1ac1-483b-a865-4715fa30d92d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:23.482130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079947cb-1ac1-483b-a865-4715fa30d92d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:23.485822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c97bb-f88e-4a58-be21-ea1b86545f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:23.486594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c97bb-f88e-4a58-be21-ea1b86545f8a service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:23.486944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c97bb-f88e-4a58-be21-ea1b86545f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:23.487289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c97bb-f88e-4a58-be21-ea1b86545f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:23.498995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c97bb-f88e-4a58-be21-ea1b86545f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:23.500704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c97bb-f88e-4a58-be21-ea1b86545f8a service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:23.513066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1134/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:58:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:58:26.430876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442111a9-39e0-4697-afc4-81aed8b4e381 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:26.441462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442111a9-39e0-4697-afc4-81aed8b4e381 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:26.445945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbce27c-af28-42c2-a202-2d652b619a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:26.446600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbce27c-af28-42c2-a202-2d652b619a5b service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:26.446899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbce27c-af28-42c2-a202-2d652b619a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:26.447170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbce27c-af28-42c2-a202-2d652b619a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:26.449284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4cbce27c-af28-42c2-a202-2d652b619a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:26.463636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbce27c-af28-42c2-a202-2d652b619a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:26.465416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbce27c-af28-42c2-a202-2d652b619a5b service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:26.477686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1140/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:58:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:58:28.445144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c97bb-f88e-4a58-be21-ea1b86545f8a service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:28.459209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c97bb-f88e-4a58-be21-ea1b86545f8a service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:28.465189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:28.465907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:28.466246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:28.466528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:28.468887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:28.483530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:28.485098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:28.497201 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1135/2275] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:58:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:58:31.091002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1e12-741d-4a27-895f-7e81a6bf6c99 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:31.091526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1e12-741d-4a27-895f-7e81a6bf6c99 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:31.091803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1e12-741d-4a27-895f-7e81a6bf6c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:31.111633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-8f8a1e12-741d-4a27-895f-7e81a6bf6c99 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:58:31.113080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8f8a1e12-741d-4a27-895f-7e81a6bf6c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:31.120953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1141/2276] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 15:58:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:58:31.133420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:31.144837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cc82e-d508-4fef-aaee-0c1ea2a9a397 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:31.150217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:31.150864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:31.151146 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:31.151417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:31.159777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:31.161373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:31.169803 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1136/2277] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:58:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:58:37.132353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99305a5-b6b2-4aff-bf28-57924b551309 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:37.133055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99305a5-b6b2-4aff-bf28-57924b551309 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:37.133055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99305a5-b6b2-4aff-bf28-57924b551309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:37.152622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d99305a5-b6b2-4aff-bf28-57924b551309 None None] MFA Rules not processed for user `32512d9ebcc8447a88b99da4181895e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:58:37.153874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d99305a5-b6b2-4aff-bf28-57924b551309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:37.223020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1142/2278] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 15:58:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:58:41.166509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=27 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:58:47.215593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:58:49.328068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:58:49.330109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:49.349520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf25ff-2bf3-43ef-9700-a248cf3e4950 service swift] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:58:49.356029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d119e-e632-447f-b3c4-74763ed1e97a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:58:49.356674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d119e-e632-447f-b3c4-74763ed1e97a service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:58:49.356949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d119e-e632-447f-b3c4-74763ed1e97a service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:58:49.357227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d119e-e632-447f-b3c4-74763ed1e97a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:58:49.359616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-688d119e-e632-447f-b3c4-74763ed1e97a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:58:49.427063 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d119e-e632-447f-b3c4-74763ed1e97a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:58:49.429503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d119e-e632-447f-b3c4-74763ed1e97a service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:58:49.441699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1137/2279] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:58:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:58:59.437072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=28 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:59:05.590920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99305a5-b6b2-4aff-bf28-57924b551309 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:59:05.601787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99305a5-b6b2-4aff-bf28-57924b551309 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:59:05.605648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:59:05.606565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:59:05.606986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:59:05.607264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:59:05.609300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:59:05.620084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:59:05.621656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:59:05.630689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1143/2280] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:59:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:59:08.776449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f167cd-4f91-4347-acb6-cbbf8010a73a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:59:08.776968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f167cd-4f91-4347-acb6-cbbf8010a73a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:59:08.776968 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f167cd-4f91-4347-acb6-cbbf8010a73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:59:08.796674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e9f167cd-4f91-4347-acb6-cbbf8010a73a None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 15:59:08.797905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e9f167cd-4f91-4347-acb6-cbbf8010a73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:59:08.806148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1138/2281] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jun 21 15:59:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 15:59:08.832357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:59:08.851484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57616c-5818-4df6-b023-284c95afa61c service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:59:08.857936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0e9bb-73ec-4732-8156-6325fd316c74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:59:08.858588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0e9bb-73ec-4732-8156-6325fd316c74 service swift] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:59:08.858874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0e9bb-73ec-4732-8156-6325fd316c74 service swift] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:59:08.859152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0e9bb-73ec-4732-8156-6325fd316c74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:59:08.867792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0e9bb-73ec-4732-8156-6325fd316c74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:59:08.869401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0e9bb-73ec-4732-8156-6325fd316c74 service swift] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:59:08.877846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1144/2282] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 15:59:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 15:59:18.791596 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:59:18.874423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=12 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 15:59:34.432162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f167cd-4f91-4347-acb6-cbbf8010a73a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 15:59:34.449245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f167cd-4f91-4347-acb6-cbbf8010a73a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 15:59:34.457192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d34603-4d99-4c8d-8b12-839162134f0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 15:59:34.458044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34603-4d99-4c8d-8b12-839162134f0f service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 15:59:34.458292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34603-4d99-4c8d-8b12-839162134f0f service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 15:59:34.458940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d34603-4d99-4c8d-8b12-839162134f0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 15:59:34.461992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b1d34603-4d99-4c8d-8b12-839162134f0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 15:59:34.476530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d34603-4d99-4c8d-8b12-839162134f0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 15:59:34.478387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d34603-4d99-4c8d-8b12-839162134f0f service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 15:59:34.491514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1139/2283] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 15:59:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 15:59:44.485256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=6 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:00:03.426573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3776c-e96d-4e0e-8bfc-4174f08db649 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:03.427190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3776c-e96d-4e0e-8bfc-4174f08db649 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:03.427190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3776c-e96d-4e0e-8bfc-4174f08db649 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:03.428501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1145/2284] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 16:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 16:00:03.439417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab8995-e744-4d74-96e1-1699154d522a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:03.439893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab8995-e744-4d74-96e1-1699154d522a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:03.440000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab8995-e744-4d74-96e1-1699154d522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:03.481322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f0ab8995-e744-4d74-96e1-1699154d522a None None] MFA Rules not processed for user `16defb911ffa485da9164fa2674baba1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:00:03.482893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f0ab8995-e744-4d74-96e1-1699154d522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:03.500318 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3776c-e96d-4e0e-8bfc-4174f08db649 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:03.502509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-51e3776c-e96d-4e0e-8bfc-4174f08db649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:03.559827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1140/2285] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:00:03.570637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab8995-e744-4d74-96e1-1699154d522a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:03.572029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f0ab8995-e744-4d74-96e1-1699154d522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:03.575811 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3776c-e96d-4e0e-8bfc-4174f08db649 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:03.582010 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:03.582891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:03.583168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:03.583447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] PATH_INFO: `/v3/users/32512d9ebcc8447a88b99da4181895e7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:03.584681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32512d9ebcc8447a88b99da4181895e7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:03.585240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab8995-e744-4d74-96e1-1699154d522a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:03.586651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:03.590802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0043994-bb6b-46f3-86ad-14c045d8805d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:03.591449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0043994-bb6b-46f3-86ad-14c045d8805d service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:03.591735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0043994-bb6b-46f3-86ad-14c045d8805d service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:03.592010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0043994-bb6b-46f3-86ad-14c045d8805d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:03.600816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0043994-bb6b-46f3-86ad-14c045d8805d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:03.602471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0043994-bb6b-46f3-86ad-14c045d8805d service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:03.613456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1141/2286] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:00:03.637355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32512d9ebcc8447a88b99da4181895e7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:03.650260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32512d9ebcc8447a88b99da4181895e7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:03.658655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32512d9ebcc8447a88b99da4181895e7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:03.659560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32512d9ebcc8447a88b99da4181895e7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:03.666558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32512d9ebcc8447a88b99da4181895e7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:03.672426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1146/2287] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:00:03 2022] DELETE /identity/v3/users/32512d9ebcc8447a88b99da4181895e7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:00:03.859707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0043994-bb6b-46f3-86ad-14c045d8805d service placement] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:03.872439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0043994-bb6b-46f3-86ad-14c045d8805d service placement] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:03.933338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:03.934249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:03.934537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:03.934820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] PATH_INFO: `/v3/projects/af3eba9c8dda4166accce40c556af46e` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:03.936056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3eba9c8dda4166accce40c556af46e)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:03.938015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:03.986944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3eba9c8dda4166accce40c556af46e', 'request_id': 'req-4eea3f6b-ed59-498a-a6c8-a6a68351c173'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:03.993306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3eba9c8dda4166accce40c556af46e', 'request_id': 'req-4eea3f6b-ed59-498a-a6c8-a6a68351c173'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:04.002401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3eba9c8dda4166accce40c556af46e', 'request_id': 'req-4eea3f6b-ed59-498a-a6c8-a6a68351c173'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:04.009935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:00:04.013934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] The token cache is being invalidate because project af3eba9c8dda4166accce40c556af46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:00:04.014371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:04.015313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:00:04.017046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1142/2288] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:00:03 2022] DELETE /identity/v3/projects/af3eba9c8dda4166accce40c556af46e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:00:13.669206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:00:27.729159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:27.730749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:27.818079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51918a1-cb5d-470a-b0e2-e6a684d62800 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:27.896348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac4c12-a056-41bc-a5da-8d8820d7f2e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:27.897017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac4c12-a056-41bc-a5da-8d8820d7f2e8 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:27.897301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac4c12-a056-41bc-a5da-8d8820d7f2e8 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:27.897580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac4c12-a056-41bc-a5da-8d8820d7f2e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:27.899827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6dac4c12-a056-41bc-a5da-8d8820d7f2e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:27.969675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac4c12-a056-41bc-a5da-8d8820d7f2e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:27.971406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac4c12-a056-41bc-a5da-8d8820d7f2e8 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:27.982340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1147/2289] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:00:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:00:37.976906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=84 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:00:41.818785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:41.820894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:41.843710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea3f6b-ed59-498a-a6c8-a6a68351c173 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:41.851138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:41.851819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:41.852111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:41.852384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:41.854559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:41.917083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:41.918726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:41.929269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1143/2290] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:00:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:00:51.924249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=28 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:00:54.008932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dae7e7-1cb3-4bf8-aae0-561404d9db54 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:54.009511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dae7e7-1cb3-4bf8-aae0-561404d9db54 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:54.009511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dae7e7-1cb3-4bf8-aae0-561404d9db54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:54.028276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-99dae7e7-1cb3-4bf8-aae0-561404d9db54 None None] MFA Rules not processed for user `75407a21b1fd46e8ba10540fc5a4c30a`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:00:54.029521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-99dae7e7-1cb3-4bf8-aae0-561404d9db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:54.088266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1148/2291] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:00:54.425230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:54.426916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:54.502019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49152-17a7-4cc0-b402-216124edfb19 service nova] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:54.506970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:54.507782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:54.508313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:54.508689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:54.510836 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:54.524179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:54.525909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:54.536190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1144/2292] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:00:54.894312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dae7e7-1cb3-4bf8-aae0-561404d9db54 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:54.895825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-99dae7e7-1cb3-4bf8-aae0-561404d9db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:54.981663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dae7e7-1cb3-4bf8-aae0-561404d9db54 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:54.986751 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3148a1a-9b4b-4c77-be36-14696af0d91c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:54.987629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148a1a-9b4b-4c77-be36-14696af0d91c service placement] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:54.988026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148a1a-9b4b-4c77-be36-14696af0d91c service placement] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:54.988511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148a1a-9b4b-4c77-be36-14696af0d91c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:54.990699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d3148a1a-9b4b-4c77-be36-14696af0d91c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:55.003205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3148a1a-9b4b-4c77-be36-14696af0d91c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:55.005272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3148a1a-9b4b-4c77-be36-14696af0d91c service placement] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:55.016311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1149/2293] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:00:57.499746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:57.515114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa5904-c05f-40d8-8fa6-e4b0997b6cb4 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:57.522962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:57.524028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:57.524360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:57.524724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:57.526688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:57.592383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:57.594286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:57.609660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1145/2294] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:00:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:00:59.151046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f052-7206-430b-ad57-e161252a680d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:59.151636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f052-7206-430b-ad57-e161252a680d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:59.151907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f052-7206-430b-ad57-e161252a680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:59.174668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6806f052-7206-430b-ad57-e161252a680d None None] MFA Rules not processed for user `9a67a5c4b7034d22b78331b907cc20bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:00:59.175941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6806f052-7206-430b-ad57-e161252a680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:59.183676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1150/2295] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:00:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:00:59.495067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:00:59.508186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b131750-b03b-4fb4-8adf-2606a2dfc80e service nova] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:00:59.512226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:00:59.512877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:00:59.513159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:00:59.513438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:00:59.515281 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:00:59.526375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:00:59.527818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:00:59.536444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1146/2296] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:00:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:01:01.508624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb10891-e03f-43f0-907a-ef08af36ea6e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:01.509198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb10891-e03f-43f0-907a-ef08af36ea6e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:01.509198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb10891-e03f-43f0-907a-ef08af36ea6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:01.527977 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2cb10891-e03f-43f0-907a-ef08af36ea6e None None] MFA Rules not processed for user `01571ac0b01845f78192269cacb23644`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:01:01.529194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2cb10891-e03f-43f0-907a-ef08af36ea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:01.537173 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1151/2297] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:01:04.605076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:04.606711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:04.623817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e9494-b79c-45a2-a373-609132c2cf36 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:04.631035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:04.631715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:04.631995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:04.632270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:04.634286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:04.648277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:04.650484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:04.662768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1147/2298] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:01:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:01:07.123400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb10891-e03f-43f0-907a-ef08af36ea6e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:07.137693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb10891-e03f-43f0-907a-ef08af36ea6e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:07.145010 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb552f-97a3-402f-9ff0-03e304445898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:07.145822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb552f-97a3-402f-9ff0-03e304445898 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:07.146191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb552f-97a3-402f-9ff0-03e304445898 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:07.146731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb552f-97a3-402f-9ff0-03e304445898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:07.149082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e3eb552f-97a3-402f-9ff0-03e304445898 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:07.163529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb552f-97a3-402f-9ff0-03e304445898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:07.165163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb552f-97a3-402f-9ff0-03e304445898 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:07.177070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1152/2299] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:01:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:01:14.657586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=66 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:01:17.171842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=69 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:01:40.465353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:40.466831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:40.488843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c8c6c-4696-4256-a3e9-aae85fd0c791 service nova] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:40.497286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:40.498503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:40.499064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:40.499649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:40.503418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:40.524834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:40.527504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:40.540821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1148/2300] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:01:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:01:46.797418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c9ccd-1cbf-4eb1-a84c-d60c6185eff6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:46.798054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c9ccd-1cbf-4eb1-a84c-d60c6185eff6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:46.798054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c9ccd-1cbf-4eb1-a84c-d60c6185eff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:46.816434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2e4c9ccd-1cbf-4eb1-a84c-d60c6185eff6 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:01:46.817641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2e4c9ccd-1cbf-4eb1-a84c-d60c6185eff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:46.880865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1153/2301] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jun 21 16:01:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:01:46.936175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:46.937987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:47.012930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3fcb-f8fb-4d5e-bacd-be5e0d108ee0 service placement] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:47.019121 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077c722-e452-41c2-b84a-21a37c71ff91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:47.019771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077c722-e452-41c2-b84a-21a37c71ff91 service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:47.020048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077c722-e452-41c2-b84a-21a37c71ff91 service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:47.020323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077c722-e452-41c2-b84a-21a37c71ff91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:47.032430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077c722-e452-41c2-b84a-21a37c71ff91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:47.035133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077c722-e452-41c2-b84a-21a37c71ff91 service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:47.046393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1149/2302] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 16:01:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 16:01:47.450505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c9ccd-1cbf-4eb1-a84c-d60c6185eff6 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:47.461446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c9ccd-1cbf-4eb1-a84c-d60c6185eff6 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:47.465463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:47.466139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:47.466426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:47.466700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:47.468584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:47.545035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:47.546577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:47.556387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1154/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:01:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:01:51.036595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077c722-e452-41c2-b84a-21a37c71ff91 service swift] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:51.047937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077c722-e452-41c2-b84a-21a37c71ff91 service swift] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:51.054140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:51.054789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:51.055056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:51.055339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:51.057486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:51.070344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:51.071952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:51.084275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1150/2304] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:01:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:01:54.428609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:54.437718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:54.456391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b15891-0043-479e-a6c8-2fe0e336c8b2 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:54.461515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:54.462547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:54.462814 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:54.463260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] PATH_INFO: `/v3/users/cc3f66be61f54ea8bac135cf7b3531dc` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:54.467944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc3f66be61f54ea8bac135cf7b3531dc)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:54.470197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:54.530172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc3f66be61f54ea8bac135cf7b3531dc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.542570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc3f66be61f54ea8bac135cf7b3531dc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.552659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc3f66be61f54ea8bac135cf7b3531dc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.553391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc3f66be61f54ea8bac135cf7b3531dc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.561495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc3f66be61f54ea8bac135cf7b3531dc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.568973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1155/2305] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:01:54 2022] DELETE /identity/v3/users/cc3f66be61f54ea8bac135cf7b3531dc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:01:54.755956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:54.771439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8f752-657d-4900-96f1-489fe66edf4d service placement] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:54.845981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:54.846819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:54.847100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:54.847365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] PATH_INFO: `/v3/projects/8716150d618c402fbb51a2a042c5ea5b` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:54.848559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] RBAC: Authorizing `identity:delete_project(project_id=8716150d618c402fbb51a2a042c5ea5b)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:54.850547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:54.905283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8716150d618c402fbb51a2a042c5ea5b', 'request_id': 'req-09f52726-50e4-4078-a522-10b3e7e2c3da'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.911854 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8716150d618c402fbb51a2a042c5ea5b', 'request_id': 'req-09f52726-50e4-4078-a522-10b3e7e2c3da'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.920883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8716150d618c402fbb51a2a042c5ea5b', 'request_id': 'req-09f52726-50e4-4078-a522-10b3e7e2c3da'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.926234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:01:54.934928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] The token cache is being invalidate because project 8716150d618c402fbb51a2a042c5ea5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:01:54.935321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.936261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:54.938067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1151/2306] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:01:54 2022] DELETE /identity/v3/projects/8716150d618c402fbb51a2a042c5ea5b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:01:54.942345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:54.943893 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:54.958388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a9970-f789-4e12-86d0-95e54743d3ff admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:54.966682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:01:55.020830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:55.021760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.022084 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.022350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] PATH_INFO: `/v3/users/c77fc86204d9404b9e3375a0fd2830d9` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.023669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77fc86204d9404b9e3375a0fd2830d9)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:55.025637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:55.083516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77fc86204d9404b9e3375a0fd2830d9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.097749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77fc86204d9404b9e3375a0fd2830d9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.106865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77fc86204d9404b9e3375a0fd2830d9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.107533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77fc86204d9404b9e3375a0fd2830d9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.115064 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f97a0228-fddb-4c64-9559-8fe27e5f0251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77fc86204d9404b9e3375a0fd2830d9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.121080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1156/2307] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:01:54 2022] DELETE /identity/v3/users/c77fc86204d9404b9e3375a0fd2830d9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:01:55.316749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:55.318323 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:55.335277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f52726-50e4-4078-a522-10b3e7e2c3da admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:55.404211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:55.404591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.404876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.405189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] PATH_INFO: `/v3/projects/fceb0977c8424b5199d1193c8630f542` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.406562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fceb0977c8424b5199d1193c8630f542)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:55.408622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:55.464408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fceb0977c8424b5199d1193c8630f542', 'request_id': 'req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.471327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fceb0977c8424b5199d1193c8630f542', 'request_id': 'req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.480743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fceb0977c8424b5199d1193c8630f542', 'request_id': 'req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.492392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] The token cache is being invalidate because project fceb0977c8424b5199d1193c8630f542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:01:55.492676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.493327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bdff2c84-2070-4350-8ed1-4523d6b3ef0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:01:55.494935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1152/2308] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:01:55 2022] DELETE /identity/v3/projects/fceb0977c8424b5199d1193c8630f542 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:01:55.500146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2a6a-b9c0-4aa1-8a1b-cef5132e698a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.500643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2a6a-b9c0-4aa1-8a1b-cef5132e698a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.500741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2a6a-b9c0-4aa1-8a1b-cef5132e698a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.521060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d7dd2a6a-b9c0-4aa1-8a1b-cef5132e698a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:01:55.522343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d7dd2a6a-b9c0-4aa1-8a1b-cef5132e698a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:55.592582 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1157/2309] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:01:55.600928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f7ae-1595-49f4-905c-f8b76f7085c3 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.601234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f7ae-1595-49f4-905c-f8b76f7085c3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.601517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f7ae-1595-49f4-905c-f8b76f7085c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.621243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a616f7ae-1595-49f4-905c-f8b76f7085c3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:01:55.622506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a616f7ae-1595-49f4-905c-f8b76f7085c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:55.631392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1153/2310] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:01:55.635748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd2a6a-b9c0-4aa1-8a1b-cef5132e698a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:55.646852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd2a6a-b9c0-4aa1-8a1b-cef5132e698a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:55.653606 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:55.654605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.654946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.655362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.656191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:01:55.656506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:55.658510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:55.677640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1158/2311] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:01:55.682598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616f7ae-1595-49f4-905c-f8b76f7085c3 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:55.696122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616f7ae-1595-49f4-905c-f8b76f7085c3 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:55.708657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb66298-f78a-404d-baee-507205da3901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:55.709396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb66298-f78a-404d-baee-507205da3901 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.709674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb66298-f78a-404d-baee-507205da3901 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.709975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb66298-f78a-404d-baee-507205da3901 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.710683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb66298-f78a-404d-baee-507205da3901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:55.712757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb66298-f78a-404d-baee-507205da3901 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:55.755439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1154/2312] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:01:55 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:01:55.759592 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:55.772803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469e4e0-6bd2-49ac-93bc-f3846baa33e4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:55.787794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:01:55.842226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3030a-fc71-413b-a076-ac24aa8e44f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:55.842927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3030a-fc71-413b-a076-ac24aa8e44f4 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.843206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3030a-fc71-413b-a076-ac24aa8e44f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.843487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3030a-fc71-413b-a076-ac24aa8e44f4 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.844236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3030a-fc71-413b-a076-ac24aa8e44f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:55.846416 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3030a-fc71-413b-a076-ac24aa8e44f4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:55.871418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1159/2313] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:01:55 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:01:55.873595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb66298-f78a-404d-baee-507205da3901 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:55.885240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb66298-f78a-404d-baee-507205da3901 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:55.892371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1b5b6-0b96-4177-ba52-ad76e52bbfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:55.893047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1b5b6-0b96-4177-ba52-ad76e52bbfab admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.893339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1b5b6-0b96-4177-ba52-ad76e52bbfab admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.893609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1b5b6-0b96-4177-ba52-ad76e52bbfab admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.894384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1b5b6-0b96-4177-ba52-ad76e52bbfab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:55.896011 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1b5b6-0b96-4177-ba52-ad76e52bbfab admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:55.906689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1155/2314] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:01:55.912289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3030a-fc71-413b-a076-ac24aa8e44f4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:55.924381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3030a-fc71-413b-a076-ac24aa8e44f4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:55.929640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b3747-6d0c-4ac9-88f2-111967021533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:55.930352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3747-6d0c-4ac9-88f2-111967021533 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.930638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3747-6d0c-4ac9-88f2-111967021533 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.930936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3747-6d0c-4ac9-88f2-111967021533 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.931676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b3747-6d0c-4ac9-88f2-111967021533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:55.933301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b3747-6d0c-4ac9-88f2-111967021533 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:55.944550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1160/2315] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:01:55.950464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1b5b6-0b96-4177-ba52-ad76e52bbfab admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:55.962726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1b5b6-0b96-4177-ba52-ad76e52bbfab admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:55.968667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:55.969587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:55.970383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:55.970687 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] PATH_INFO: `/v3/projects/658a35680c70498db2bccc1085c22b21/users/0943474b3d4d49d7973619388dab2be1/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:55.988472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=658a35680c70498db2bccc1085c22b21, user_id=0943474b3d4d49d7973619388dab2be1, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:55.992516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:56.015075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1156/2316] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:01:55 2022] PUT /identity/v3/projects/658a35680c70498db2bccc1085c22b21/users/0943474b3d4d49d7973619388dab2be1/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:01:56.023463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c7faa-17a7-4d1c-8654-b13ccd16df0e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:56.023792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c7faa-17a7-4d1c-8654-b13ccd16df0e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:56.024256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c7faa-17a7-4d1c-8654-b13ccd16df0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:56.056768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2c6c7faa-17a7-4d1c-8654-b13ccd16df0e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:01:56.058045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2c6c7faa-17a7-4d1c-8654-b13ccd16df0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:56.120867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1161/2317] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:01:56.131275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:01:56.133199 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:01:56.147748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:01:56.200758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50001b-2200-40ad-9018-9317808dfa9c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:01:56.204850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cdef8-63ff-4e87-9c74-ede7503e2306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:01:56.205502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cdef8-63ff-4e87-9c74-ede7503e2306 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:01:56.205819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cdef8-63ff-4e87-9c74-ede7503e2306 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:01:56.206131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cdef8-63ff-4e87-9c74-ede7503e2306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:01:56.216690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cdef8-63ff-4e87-9c74-ede7503e2306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:01:56.218235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cdef8-63ff-4e87-9c74-ede7503e2306 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:01:56.228633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1157/2318] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:02:00.229349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f6f0c-d501-495b-91d6-75477435ce2d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:00.229924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f6f0c-d501-495b-91d6-75477435ce2d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:00.230022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f6f0c-d501-495b-91d6-75477435ce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:00.271196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2d8f6f0c-d501-495b-91d6-75477435ce2d None None] MFA Rules not processed for user `0943474b3d4d49d7973619388dab2be1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:00.272429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2d8f6f0c-d501-495b-91d6-75477435ce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:00.344347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1162/2319] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:00.348829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cdef8-63ff-4e87-9c74-ede7503e2306 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:00.359045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cdef8-63ff-4e87-9c74-ede7503e2306 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:00.364403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe929bb-24b6-4dd3-93fd-6ab4e5e1ad67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:00.365072 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe929bb-24b6-4dd3-93fd-6ab4e5e1ad67 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:00.365351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe929bb-24b6-4dd3-93fd-6ab4e5e1ad67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:00.365656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe929bb-24b6-4dd3-93fd-6ab4e5e1ad67 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:00.366390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe929bb-24b6-4dd3-93fd-6ab4e5e1ad67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:00.368330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe929bb-24b6-4dd3-93fd-6ab4e5e1ad67 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:00.397168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1158/2320] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:02:00 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:02:00.401446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f6f0c-d501-495b-91d6-75477435ce2d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:00.413376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f6f0c-d501-495b-91d6-75477435ce2d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:00.444234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:00.491081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267ddd-d2e4-41b0-b8ec-e8228d6a5dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:00.492107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267ddd-d2e4-41b0-b8ec-e8228d6a5dbe admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:00.492464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267ddd-d2e4-41b0-b8ec-e8228d6a5dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:00.492971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0267ddd-d2e4-41b0-b8ec-e8228d6a5dbe admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:00.493693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0267ddd-d2e4-41b0-b8ec-e8228d6a5dbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:00.495693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0267ddd-d2e4-41b0-b8ec-e8228d6a5dbe admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:00.532922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1163/2321] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:02:00 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:02:00.534292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe929bb-24b6-4dd3-93fd-6ab4e5e1ad67 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:00.550529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe929bb-24b6-4dd3-93fd-6ab4e5e1ad67 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:00.555592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b5288-bc29-4e18-95dc-721607c56495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:00.556288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5288-bc29-4e18-95dc-721607c56495 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:00.556571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5288-bc29-4e18-95dc-721607c56495 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:00.556842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5288-bc29-4e18-95dc-721607c56495 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:00.557577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b5288-bc29-4e18-95dc-721607c56495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:00.559318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b5288-bc29-4e18-95dc-721607c56495 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:00.572177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1159/2322] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:02:00.576338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267ddd-d2e4-41b0-b8ec-e8228d6a5dbe admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:00.587858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0267ddd-d2e4-41b0-b8ec-e8228d6a5dbe admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:00.593066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617667f-ce62-4de8-aa11-c24f1586fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:00.594005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617667f-ce62-4de8-aa11-c24f1586fcd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:00.594351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617667f-ce62-4de8-aa11-c24f1586fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:00.594622 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617667f-ce62-4de8-aa11-c24f1586fcd7 admin admin] PATH_INFO: `/v3/projects/983d1968d3a24c1882a8973cbd495e36/users/0b8ab25c4b3b435f86b85c219a908f71/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:00.611057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617667f-ce62-4de8-aa11-c24f1586fcd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983d1968d3a24c1882a8973cbd495e36, user_id=0b8ab25c4b3b435f86b85c219a908f71, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:00.613924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617667f-ce62-4de8-aa11-c24f1586fcd7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:00.630919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1164/2323] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:02:00 2022] PUT /identity/v3/projects/983d1968d3a24c1882a8973cbd495e36/users/0b8ab25c4b3b435f86b85c219a908f71/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:02:00.634826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b5288-bc29-4e18-95dc-721607c56495 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:00.646542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b5288-bc29-4e18-95dc-721607c56495 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:00.715724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed8e7f-8c29-4228-b418-70a1cb422ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:00.716433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8e7f-8c29-4228-b418-70a1cb422ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:00.716712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8e7f-8c29-4228-b418-70a1cb422ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:00.716987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed8e7f-8c29-4228-b418-70a1cb422ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:00.717719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed8e7f-8c29-4228-b418-70a1cb422ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:00.719501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed8e7f-8c29-4228-b418-70a1cb422ab6 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:00.733957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1160/2324] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:02:00.736101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617667f-ce62-4de8-aa11-c24f1586fcd7 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:00.748524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617667f-ce62-4de8-aa11-c24f1586fcd7 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:00.754357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:00.755295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:00.755596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:00.755883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] PATH_INFO: `/v3/projects/983d1968d3a24c1882a8973cbd495e36/users/0b8ab25c4b3b435f86b85c219a908f71/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:00.758034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983d1968d3a24c1882a8973cbd495e36, user_id=0b8ab25c4b3b435f86b85c219a908f71, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:00.760725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:00.772220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1165/2325] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:02:00 2022] PUT /identity/v3/projects/983d1968d3a24c1882a8973cbd495e36/users/0b8ab25c4b3b435f86b85c219a908f71/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:02:05.497316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66be08-1d8d-4e0d-aa63-64624b70c0a7 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:05.498274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66be08-1d8d-4e0d-aa63-64624b70c0a7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:05.499365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66be08-1d8d-4e0d-aa63-64624b70c0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:05.533684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ad66be08-1d8d-4e0d-aa63-64624b70c0a7 None None] MFA Rules not processed for user `0b8ab25c4b3b435f86b85c219a908f71`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:05.534970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ad66be08-1d8d-4e0d-aa63-64624b70c0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:05.635951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1161/2326] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:05.652748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:05.655558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:05.789430 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2d484-00e1-49c0-a60b-a9574547be92 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:05.796197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52802c-eba5-4edd-8f8a-e619368f169a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:05.796918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52802c-eba5-4edd-8f8a-e619368f169a service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:05.797202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52802c-eba5-4edd-8f8a-e619368f169a service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:05.797474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52802c-eba5-4edd-8f8a-e619368f169a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:05.862380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52802c-eba5-4edd-8f8a-e619368f169a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:05.864001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52802c-eba5-4edd-8f8a-e619368f169a service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:05.873148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1166/2327] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:02:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:02:12.091610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047cf23-8c11-4b4a-9f4c-faef54ac95e3 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:12.092167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047cf23-8c11-4b4a-9f4c-faef54ac95e3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:12.092266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047cf23-8c11-4b4a-9f4c-faef54ac95e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:12.098908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:12.113003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-5047cf23-8c11-4b4a-9f4c-faef54ac95e3 None None] MFA Rules not processed for user `fa5a48626c934dc2aa4f24002970b6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:12.114242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5047cf23-8c11-4b4a-9f4c-faef54ac95e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:12.179107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1162/2328] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:15.869119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:22.171895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=24 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:30.824406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb46f0-66e2-4d7d-8c4c-9e2d94ad278f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:30.824999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb46f0-66e2-4d7d-8c4c-9e2d94ad278f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:30.824999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb46f0-66e2-4d7d-8c4c-9e2d94ad278f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:30.842955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-82bb46f0-66e2-4d7d-8c4c-9e2d94ad278f None None] MFA Rules not processed for user `0943474b3d4d49d7973619388dab2be1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:30.844179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-82bb46f0-66e2-4d7d-8c4c-9e2d94ad278f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:30.855703 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1167/2329] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:40.838073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:44.400917 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047cf23-8c11-4b4a-9f4c-faef54ac95e3 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:44.475195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047cf23-8c11-4b4a-9f4c-faef54ac95e3 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:44.479343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:44.479995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:44.480283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:44.480565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:44.482495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:44.548267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:44.549647 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:44.559947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1163/2330] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:02:46.476633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb46f0-66e2-4d7d-8c4c-9e2d94ad278f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:46.478510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-82bb46f0-66e2-4d7d-8c4c-9e2d94ad278f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:46.495407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb46f0-66e2-4d7d-8c4c-9e2d94ad278f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:46.517022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daceb3-f73b-4f47-a901-37e8de681215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:46.518196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daceb3-f73b-4f47-a901-37e8de681215 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:46.518986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daceb3-f73b-4f47-a901-37e8de681215 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:46.519263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daceb3-f73b-4f47-a901-37e8de681215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:46.531460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0daceb3-f73b-4f47-a901-37e8de681215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:46.533170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0daceb3-f73b-4f47-a901-37e8de681215 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:46.546404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1168/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:02:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:02:47.998766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:48.000459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:48.017025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d784f-9888-438b-9f1a-9b44e25e11e5 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:48.029066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:48.030188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.030460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.030727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] PATH_INFO: `/v3/users/01571ac0b01845f78192269cacb23644` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.032022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] RBAC: Authorizing `identity:delete_user(user_id=01571ac0b01845f78192269cacb23644)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:48.034134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:48.089459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01571ac0b01845f78192269cacb23644'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.103718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01571ac0b01845f78192269cacb23644'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.114936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01571ac0b01845f78192269cacb23644'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.123025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01571ac0b01845f78192269cacb23644'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.123743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-60908ddf-e058-447f-8732-13f56b04358f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01571ac0b01845f78192269cacb23644'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.144714 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1164/2332] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:02:47 2022] DELETE /identity/v3/users/01571ac0b01845f78192269cacb23644 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:02:48.335338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daceb3-f73b-4f47-a901-37e8de681215 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:48.346646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daceb3-f73b-4f47-a901-37e8de681215 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:48.453585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:48.454648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.455005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.455572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] PATH_INFO: `/v3/projects/8818885318b54ef6858e754d2bc4587c` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.456570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8818885318b54ef6858e754d2bc4587c)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:48.460196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:48.526369 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8818885318b54ef6858e754d2bc4587c', 'request_id': 'req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.539857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8818885318b54ef6858e754d2bc4587c', 'request_id': 'req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.549099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8818885318b54ef6858e754d2bc4587c', 'request_id': 'req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.564103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] The token cache is being invalidate because project 8818885318b54ef6858e754d2bc4587c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:02:48.564968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.565087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f2c4b0af-2b75-4070-ab8c-0e8b5cc61648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:48.567857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1169/2333] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:02:48 2022] DELETE /identity/v3/projects/8818885318b54ef6858e754d2bc4587c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:02:48.573119 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840054-b3f6-4a9a-9d60-1c5cb21eb99d None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.573515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840054-b3f6-4a9a-9d60-1c5cb21eb99d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.573681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f840054-b3f6-4a9a-9d60-1c5cb21eb99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.593521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-5f840054-b3f6-4a9a-9d60-1c5cb21eb99d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:48.594970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5f840054-b3f6-4a9a-9d60-1c5cb21eb99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:48.619658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:48.678548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1165/2334] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:48.681516 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445424b6-ccf6-4f29-8ffd-056542438a14 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.681829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445424b6-ccf6-4f29-8ffd-056542438a14 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.682106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445424b6-ccf6-4f29-8ffd-056542438a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.701822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-445424b6-ccf6-4f29-8ffd-056542438a14 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:48.703122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-445424b6-ccf6-4f29-8ffd-056542438a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:48.715954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1170/2335] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:48.717508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840054-b3f6-4a9a-9d60-1c5cb21eb99d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:48.728998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f840054-b3f6-4a9a-9d60-1c5cb21eb99d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:48.733594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:48.735862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.735862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.735862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.735862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:02:48.735862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:48.737649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:48.753059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1166/2336] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 16:02:48.756891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445424b6-ccf6-4f29-8ffd-056542438a14 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:48.775269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445424b6-ccf6-4f29-8ffd-056542438a14 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:48.781310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e5415-f5f5-484e-bad7-6454cb3cbac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:48.782025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e5415-f5f5-484e-bad7-6454cb3cbac0 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.782367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e5415-f5f5-484e-bad7-6454cb3cbac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.782725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e5415-f5f5-484e-bad7-6454cb3cbac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.783490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e5415-f5f5-484e-bad7-6454cb3cbac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:48.785983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e5415-f5f5-484e-bad7-6454cb3cbac0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:48.818041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1171/2337] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:02:48 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:02:48.822127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:48.833143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74f72a-6466-4092-8db2-1597af937f3e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:48.898320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b5acd-cf79-4a4b-951a-c3ea67e9fe2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:48.899568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b5acd-cf79-4a4b-951a-c3ea67e9fe2b admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.899568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b5acd-cf79-4a4b-951a-c3ea67e9fe2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.899568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b5acd-cf79-4a4b-951a-c3ea67e9fe2b admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.900346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b5acd-cf79-4a4b-951a-c3ea67e9fe2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:48.902616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b5acd-cf79-4a4b-951a-c3ea67e9fe2b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:48.928405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1167/2338] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:02:48 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:02:48.931709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e5415-f5f5-484e-bad7-6454cb3cbac0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:48.947286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e5415-f5f5-484e-bad7-6454cb3cbac0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:48.951973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff3fe-c585-4bf5-9761-95f8a838a2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:48.952701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff3fe-c585-4bf5-9761-95f8a838a2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.952994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff3fe-c585-4bf5-9761-95f8a838a2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.953265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cff3fe-c585-4bf5-9761-95f8a838a2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.954037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cff3fe-c585-4bf5-9761-95f8a838a2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:48.955758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cff3fe-c585-4bf5-9761-95f8a838a2f0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:48.969893 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1172/2339] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:02:48.974743 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b5acd-cf79-4a4b-951a-c3ea67e9fe2b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:48.988168 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b5acd-cf79-4a4b-951a-c3ea67e9fe2b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:48.993637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710b4ef-7562-4762-9658-8138f13d2474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:48.994895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710b4ef-7562-4762-9658-8138f13d2474 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:48.995183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710b4ef-7562-4762-9658-8138f13d2474 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:48.995459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710b4ef-7562-4762-9658-8138f13d2474 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:48.996225 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710b4ef-7562-4762-9658-8138f13d2474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:48.997901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710b4ef-7562-4762-9658-8138f13d2474 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:49.012856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1168/2340] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:02:49.014795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff3fe-c585-4bf5-9761-95f8a838a2f0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:49.026637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cff3fe-c585-4bf5-9761-95f8a838a2f0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:49.031922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:49.032771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:49.033052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:49.033331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] PATH_INFO: `/v3/projects/982ffab40b084e389a6a101e485a12a5/users/343c8b40c0444171874f196c600b4cfe/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:49.050113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=982ffab40b084e389a6a101e485a12a5, user_id=343c8b40c0444171874f196c600b4cfe, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:49.052868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:49.070683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1173/2341] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:02:49 2022] PUT /identity/v3/projects/982ffab40b084e389a6a101e485a12a5/users/343c8b40c0444171874f196c600b4cfe/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:02:49.081955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361291-f085-45a3-be2d-d402dc75968a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:49.081955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361291-f085-45a3-be2d-d402dc75968a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:49.082766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361291-f085-45a3-be2d-d402dc75968a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:49.114413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c3361291-f085-45a3-be2d-d402dc75968a None None] MFA Rules not processed for user `343c8b40c0444171874f196c600b4cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:49.115727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c3361291-f085-45a3-be2d-d402dc75968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:49.192407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1169/2342] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:49.204178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:49.206031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:49.221882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:49.270572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f943f-0ac0-47b7-a2c3-f52eac160260 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:49.275109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b4cda-302d-495b-978e-33452abd1290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:49.275879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b4cda-302d-495b-978e-33452abd1290 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:49.276156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b4cda-302d-495b-978e-33452abd1290 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:49.276419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b4cda-302d-495b-978e-33452abd1290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:49.286270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b4cda-302d-495b-978e-33452abd1290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:49.287550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b4cda-302d-495b-978e-33452abd1290 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:49.297947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1174/2343] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:02:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:02:57.821360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3361291-f085-45a3-be2d-d402dc75968a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:57.848105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:57.898856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3361291-f085-45a3-be2d-d402dc75968a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:57.904103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:57.904977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:57.905254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:57.905541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] PATH_INFO: `/v3/users/343c8b40c0444171874f196c600b4cfe` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:57.906831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=343c8b40c0444171874f196c600b4cfe)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:57.908762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:57.966557 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343c8b40c0444171874f196c600b4cfe'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:57.979961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343c8b40c0444171874f196c600b4cfe'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:57.989554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343c8b40c0444171874f196c600b4cfe'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:57.997295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343c8b40c0444171874f196c600b4cfe'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:57.998056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343c8b40c0444171874f196c600b4cfe'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:58.004416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1170/2344] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:02:57 2022] DELETE /identity/v3/users/343c8b40c0444171874f196c600b4cfe => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:02:58.011424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71ea6-6adc-4dcf-a7f1-b47df0976abb None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.012077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71ea6-6adc-4dcf-a7f1-b47df0976abb None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.012077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71ea6-6adc-4dcf-a7f1-b47df0976abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.036290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e4a71ea6-6adc-4dcf-a7f1-b47df0976abb None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:58.037904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e4a71ea6-6adc-4dcf-a7f1-b47df0976abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:58.114670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1175/2345] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:58.126941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:58.128684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:58.194340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccef50c-25c2-4126-8873-4c1b4e2e90f5 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:58.198497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0629c-147d-48a5-a0f5-29f79d5826ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:58.199144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0629c-147d-48a5-a0f5-29f79d5826ca service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.199421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0629c-147d-48a5-a0f5-29f79d5826ca service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.199695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0629c-147d-48a5-a0f5-29f79d5826ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.210220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0629c-147d-48a5-a0f5-29f79d5826ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:58.211668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0629c-147d-48a5-a0f5-29f79d5826ca service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:58.221342 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1171/2346] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:02:58.510456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71ea6-6adc-4dcf-a7f1-b47df0976abb None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:58.512171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e4a71ea6-6adc-4dcf-a7f1-b47df0976abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:58.526980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71ea6-6adc-4dcf-a7f1-b47df0976abb None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:58.532226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:58.533049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.533357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.533641 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] PATH_INFO: `/v3/projects/982ffab40b084e389a6a101e485a12a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.534934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=982ffab40b084e389a6a101e485a12a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:58.538339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:58.590847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '982ffab40b084e389a6a101e485a12a5', 'request_id': 'req-7ba76731-8b64-4bce-b585-f6a99949d024'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:58.599831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '982ffab40b084e389a6a101e485a12a5', 'request_id': 'req-7ba76731-8b64-4bce-b585-f6a99949d024'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:58.606922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '982ffab40b084e389a6a101e485a12a5', 'request_id': 'req-7ba76731-8b64-4bce-b585-f6a99949d024'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:58.618186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] The token cache is being invalidate because project 982ffab40b084e389a6a101e485a12a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:02:58.618495 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:58.619238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7ba76731-8b64-4bce-b585-f6a99949d024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:02:58.620878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1176/2347] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:02:58 2022] DELETE /identity/v3/projects/982ffab40b084e389a6a101e485a12a5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:02:58.628215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c563d6-9dc9-4259-af6e-79537a207c70 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.628500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c563d6-9dc9-4259-af6e-79537a207c70 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.628776 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c563d6-9dc9-4259-af6e-79537a207c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.647724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d7c563d6-9dc9-4259-af6e-79537a207c70 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:58.648915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d7c563d6-9dc9-4259-af6e-79537a207c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:58.686480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:58.716223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1172/2348] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:58.724225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba83614-49a7-4a47-a040-3d45e7399511 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.724581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba83614-49a7-4a47-a040-3d45e7399511 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.724923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba83614-49a7-4a47-a040-3d45e7399511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.743688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-aba83614-49a7-4a47-a040-3d45e7399511 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:58.745044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-aba83614-49a7-4a47-a040-3d45e7399511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:58.753110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1177/2349] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:58.757927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c563d6-9dc9-4259-af6e-79537a207c70 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:58.767933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c563d6-9dc9-4259-af6e-79537a207c70 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:58.772864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:58.773524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.773832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.774117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.774846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:02:58.775152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:58.776894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:58.789304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1173/2350] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:02:58.793890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba83614-49a7-4a47-a040-3d45e7399511 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:58.804945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba83614-49a7-4a47-a040-3d45e7399511 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:58.810936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37f2a5-e20e-436f-8839-1f84237d7f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:58.811602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37f2a5-e20e-436f-8839-1f84237d7f4a admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.811886 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37f2a5-e20e-436f-8839-1f84237d7f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.812182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37f2a5-e20e-436f-8839-1f84237d7f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.812883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37f2a5-e20e-436f-8839-1f84237d7f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:58.814834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37f2a5-e20e-436f-8839-1f84237d7f4a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:58.849082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1178/2351] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:02:58 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:02:58.855592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:58.871484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bf7ab-ecf7-4a68-8a98-ebe63c5a03bd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:58.936281 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a895-8612-403d-85ad-60bcfaffa5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:58.936940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a895-8612-403d-85ad-60bcfaffa5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.937232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a895-8612-403d-85ad-60bcfaffa5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.937504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0a895-8612-403d-85ad-60bcfaffa5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.938232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a895-8612-403d-85ad-60bcfaffa5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:58.940183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0a895-8612-403d-85ad-60bcfaffa5b1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:58.963476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1174/2352] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:02:58 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:02:58.967738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37f2a5-e20e-436f-8839-1f84237d7f4a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:58.978905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37f2a5-e20e-436f-8839-1f84237d7f4a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:58.984108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e791e-6383-4f20-a536-784946e14914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:58.984738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e791e-6383-4f20-a536-784946e14914 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:58.985017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e791e-6383-4f20-a536-784946e14914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:58.985289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e791e-6383-4f20-a536-784946e14914 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:58.986065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e791e-6383-4f20-a536-784946e14914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:58.987666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e791e-6383-4f20-a536-784946e14914 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:58.993306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:58.998841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1179/2353] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:02:59.003000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a895-8612-403d-85ad-60bcfaffa5b1 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.013418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0a895-8612-403d-85ad-60bcfaffa5b1 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.018506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1e45b-3229-4b30-9f28-496a36692e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.019164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1e45b-3229-4b30-9f28-496a36692e2b admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.019448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1e45b-3229-4b30-9f28-496a36692e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.019727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1e45b-3229-4b30-9f28-496a36692e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.020481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1e45b-3229-4b30-9f28-496a36692e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.022133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1e45b-3229-4b30-9f28-496a36692e2b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.032197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1175/2354] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:02:59.038328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e791e-6383-4f20-a536-784946e14914 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.048749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e791e-6383-4f20-a536-784946e14914 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.054160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9f2e2-6e6e-4182-8ec3-427ac41a0dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.055070 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9f2e2-6e6e-4182-8ec3-427ac41a0dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.055349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9f2e2-6e6e-4182-8ec3-427ac41a0dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.055621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9f2e2-6e6e-4182-8ec3-427ac41a0dc4 admin admin] PATH_INFO: `/v3/projects/8796bc0d66ed4fa0bb98a23227793ef2/users/fafa652df2ed40cca1792c8ce2397c9a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.072638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9f2e2-6e6e-4182-8ec3-427ac41a0dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8796bc0d66ed4fa0bb98a23227793ef2, user_id=fafa652df2ed40cca1792c8ce2397c9a, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.075350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9f2e2-6e6e-4182-8ec3-427ac41a0dc4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.090312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1180/2355] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:02:59 2022] PUT /identity/v3/projects/8796bc0d66ed4fa0bb98a23227793ef2/users/fafa652df2ed40cca1792c8ce2397c9a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:02:59.103128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5af521-7c00-4a83-922b-7aabf1d0d923 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.103430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5af521-7c00-4a83-922b-7aabf1d0d923 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.103742 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5af521-7c00-4a83-922b-7aabf1d0d923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.135166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-bd5af521-7c00-4a83-922b-7aabf1d0d923 None None] MFA Rules not processed for user `fafa652df2ed40cca1792c8ce2397c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:59.136399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bd5af521-7c00-4a83-922b-7aabf1d0d923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:59.206009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1176/2356] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:02:59.211200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9f2e2-6e6e-4182-8ec3-427ac41a0dc4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.278572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9f2e2-6e6e-4182-8ec3-427ac41a0dc4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.283576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b1a7-0d04-42fd-85d6-3db328a5a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.284272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b1a7-0d04-42fd-85d6-3db328a5a4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.284550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b1a7-0d04-42fd-85d6-3db328a5a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.284821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b1a7-0d04-42fd-85d6-3db328a5a4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.285550 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8b1a7-0d04-42fd-85d6-3db328a5a4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.287680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8b1a7-0d04-42fd-85d6-3db328a5a4b5 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.314684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1181/2357] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:02:59 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:02:59.321032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5af521-7c00-4a83-922b-7aabf1d0d923 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.331961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5af521-7c00-4a83-922b-7aabf1d0d923 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.365794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:59.395688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a6644-57df-4979-bd61-b1b9bd7f4903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.396349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6644-57df-4979-bd61-b1b9bd7f4903 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.396652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6644-57df-4979-bd61-b1b9bd7f4903 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.396999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6644-57df-4979-bd61-b1b9bd7f4903 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.397666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6644-57df-4979-bd61-b1b9bd7f4903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.399680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6644-57df-4979-bd61-b1b9bd7f4903 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.422482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1177/2358] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:02:59 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:02:59.426951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b1a7-0d04-42fd-85d6-3db328a5a4b5 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.440404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b1a7-0d04-42fd-85d6-3db328a5a4b5 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.450063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c965d50-8866-4c80-a8c9-0fbc1816c93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.451065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c965d50-8866-4c80-a8c9-0fbc1816c93b admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.451447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c965d50-8866-4c80-a8c9-0fbc1816c93b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.451779 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c965d50-8866-4c80-a8c9-0fbc1816c93b admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.452849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c965d50-8866-4c80-a8c9-0fbc1816c93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.455338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c965d50-8866-4c80-a8c9-0fbc1816c93b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.469477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1182/2359] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:02:59.476343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a6644-57df-4979-bd61-b1b9bd7f4903 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.487209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a6644-57df-4979-bd61-b1b9bd7f4903 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.494316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6eaa8-3df7-47ca-9c2e-c80cc11c61c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.495237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6eaa8-3df7-47ca-9c2e-c80cc11c61c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.495529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6eaa8-3df7-47ca-9c2e-c80cc11c61c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.495790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6eaa8-3df7-47ca-9c2e-c80cc11c61c8 admin admin] PATH_INFO: `/v3/projects/a1c44243fce04b97a7c23551adadb29a/users/145047d1e3934b65a91b4bfa6a20e399/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.512519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6eaa8-3df7-47ca-9c2e-c80cc11c61c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c44243fce04b97a7c23551adadb29a, user_id=145047d1e3934b65a91b4bfa6a20e399, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.515172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6eaa8-3df7-47ca-9c2e-c80cc11c61c8 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.530219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1178/2360] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:02:59 2022] PUT /identity/v3/projects/a1c44243fce04b97a7c23551adadb29a/users/145047d1e3934b65a91b4bfa6a20e399/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:02:59.535519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c965d50-8866-4c80-a8c9-0fbc1816c93b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.547105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c965d50-8866-4c80-a8c9-0fbc1816c93b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.619442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c764563-f117-420a-aba8-94c127bf7d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.620188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c764563-f117-420a-aba8-94c127bf7d70 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.620470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c764563-f117-420a-aba8-94c127bf7d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.620746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c764563-f117-420a-aba8-94c127bf7d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.621710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c764563-f117-420a-aba8-94c127bf7d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.623156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c764563-f117-420a-aba8-94c127bf7d70 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.637332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1183/2361] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:02:59.638599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6eaa8-3df7-47ca-9c2e-c80cc11c61c8 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.649802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6eaa8-3df7-47ca-9c2e-c80cc11c61c8 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.654985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.655868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.656150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.656438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] PATH_INFO: `/v3/projects/a1c44243fce04b97a7c23551adadb29a/users/145047d1e3934b65a91b4bfa6a20e399/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.658479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c44243fce04b97a7c23551adadb29a, user_id=145047d1e3934b65a91b4bfa6a20e399, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.661102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.673257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1179/2362] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:02:59 2022] PUT /identity/v3/projects/a1c44243fce04b97a7c23551adadb29a/users/145047d1e3934b65a91b4bfa6a20e399/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:02:59.683493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dd0bd-7a44-4e19-905d-94483807cd0a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.683805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dd0bd-7a44-4e19-905d-94483807cd0a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.684092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dd0bd-7a44-4e19-905d-94483807cd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.712647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-191dd0bd-7a44-4e19-905d-94483807cd0a None None] MFA Rules not processed for user `145047d1e3934b65a91b4bfa6a20e399`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:02:59.713898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-191dd0bd-7a44-4e19-905d-94483807cd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:59.751336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:02:59.787575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1184/2363] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:02:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:02:59.799960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:02:59.802438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:02:59.867909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902850fa-e88b-4bb9-b589-4fab4f990363 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:02:59.871934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396306f-cd4b-48a9-9a3a-bdc3198764f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:02:59.872601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396306f-cd4b-48a9-9a3a-bdc3198764f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:02:59.872884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396306f-cd4b-48a9-9a3a-bdc3198764f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:02:59.873143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396306f-cd4b-48a9-9a3a-bdc3198764f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:02:59.882983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396306f-cd4b-48a9-9a3a-bdc3198764f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:02:59.884430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396306f-cd4b-48a9-9a3a-bdc3198764f6 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:02:59.894235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1180/2364] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:02:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:02:59.969523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dd0bd-7a44-4e19-905d-94483807cd0a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:00.036587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dd0bd-7a44-4e19-905d-94483807cd0a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:00.041655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:00.042713 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:00.042981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:00.043248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] PATH_INFO: `/v3/users/fafa652df2ed40cca1792c8ce2397c9a` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:00.044583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafa652df2ed40cca1792c8ce2397c9a)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:00.046570 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:00.098877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafa652df2ed40cca1792c8ce2397c9a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.111846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafa652df2ed40cca1792c8ce2397c9a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.122073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafa652df2ed40cca1792c8ce2397c9a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.122920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafa652df2ed40cca1792c8ce2397c9a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.129799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafa652df2ed40cca1792c8ce2397c9a'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.136133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1185/2365] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:02:59 2022] DELETE /identity/v3/users/fafa652df2ed40cca1792c8ce2397c9a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:00.140790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3432-1b5e-4117-9481-2cb6f31d6394 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:00.141147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3432-1b5e-4117-9481-2cb6f31d6394 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:00.141491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb3432-1b5e-4117-9481-2cb6f31d6394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:00.160721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-97eb3432-1b5e-4117-9481-2cb6f31d6394 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:00.161973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-97eb3432-1b5e-4117-9481-2cb6f31d6394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:00.225651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1181/2366] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:00.241013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:00.242741 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:00.305689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388eb6a-ca25-48bd-9436-c783617fb728 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:00.310186 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ab7cc-33d3-4792-8e04-6d9b382e7ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:00.310832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab7cc-33d3-4792-8e04-6d9b382e7ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:00.311110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab7cc-33d3-4792-8e04-6d9b382e7ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:00.311382 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab7cc-33d3-4792-8e04-6d9b382e7ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:00.317728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:00.323368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ab7cc-33d3-4792-8e04-6d9b382e7ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:00.325082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ab7cc-33d3-4792-8e04-6d9b382e7ca1 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:00.336759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1186/2367] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:00.642534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb3432-1b5e-4117-9481-2cb6f31d6394 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:00.644034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-97eb3432-1b5e-4117-9481-2cb6f31d6394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:00.656549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb3432-1b5e-4117-9481-2cb6f31d6394 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:00.661504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:00.662498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:00.662941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:00.663219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] PATH_INFO: `/v3/projects/8796bc0d66ed4fa0bb98a23227793ef2` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:00.664526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8796bc0d66ed4fa0bb98a23227793ef2)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:00.666514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:00.700880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:00.720624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8796bc0d66ed4fa0bb98a23227793ef2', 'request_id': 'req-cb1c3b47-63b0-4e25-9a31-7adb1680444b'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.727065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8796bc0d66ed4fa0bb98a23227793ef2', 'request_id': 'req-cb1c3b47-63b0-4e25-9a31-7adb1680444b'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.734728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8796bc0d66ed4fa0bb98a23227793ef2', 'request_id': 'req-cb1c3b47-63b0-4e25-9a31-7adb1680444b'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.748971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] The token cache is being invalidate because project 8796bc0d66ed4fa0bb98a23227793ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:00.749297 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.750060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.752319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1182/2368] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:00 2022] DELETE /identity/v3/projects/8796bc0d66ed4fa0bb98a23227793ef2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:00.758372 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ab7cc-33d3-4792-8e04-6d9b382e7ca1 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:00.773317 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ab7cc-33d3-4792-8e04-6d9b382e7ca1 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:00.842326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:00.843138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:00.843419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:00.843687 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] PATH_INFO: `/v3/users/145047d1e3934b65a91b4bfa6a20e399` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:00.844918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=145047d1e3934b65a91b4bfa6a20e399)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:00.846947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:00.901348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '145047d1e3934b65a91b4bfa6a20e399'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.918997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '145047d1e3934b65a91b4bfa6a20e399'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.928244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '145047d1e3934b65a91b4bfa6a20e399'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.929480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '145047d1e3934b65a91b4bfa6a20e399'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.938596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-6e8f6df5-fbd3-4c9d-8479-fc97b3126f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '145047d1e3934b65a91b4bfa6a20e399'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:00.947167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1187/2369] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:03:00 2022] DELETE /identity/v3/users/145047d1e3934b65a91b4bfa6a20e399 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:01.242180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:01.247873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:01.262033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c3b47-63b0-4e25-9a31-7adb1680444b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:01.326344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:01.327222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.327502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.327774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] PATH_INFO: `/v3/projects/a1c44243fce04b97a7c23551adadb29a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.329053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c44243fce04b97a7c23551adadb29a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:01.331043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:01.381375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c44243fce04b97a7c23551adadb29a', 'request_id': 'req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:01.387738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c44243fce04b97a7c23551adadb29a', 'request_id': 'req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:01.397179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c44243fce04b97a7c23551adadb29a', 'request_id': 'req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:01.413216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] The token cache is being invalidate because project a1c44243fce04b97a7c23551adadb29a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:01.413504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:01.414161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7d33c4b4-07f7-4e13-8f84-f6c38e685d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:01.415717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1183/2370] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:01 2022] DELETE /identity/v3/projects/a1c44243fce04b97a7c23551adadb29a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:01.420842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18887e3d-1cc2-476d-8e80-ad57c47304fa None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.421328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18887e3d-1cc2-476d-8e80-ad57c47304fa None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.421427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18887e3d-1cc2-476d-8e80-ad57c47304fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.439929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-18887e3d-1cc2-476d-8e80-ad57c47304fa None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:01.441125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-18887e3d-1cc2-476d-8e80-ad57c47304fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:01.511580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1188/2371] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:01.519897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dcdb3-2082-4821-9417-36a899e371f0 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.520182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dcdb3-2082-4821-9417-36a899e371f0 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.520466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dcdb3-2082-4821-9417-36a899e371f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.541588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-127dcdb3-2082-4821-9417-36a899e371f0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:01.542848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-127dcdb3-2082-4821-9417-36a899e371f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:01.552289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1184/2372] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:01.557426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18887e3d-1cc2-476d-8e80-ad57c47304fa None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:01.567763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18887e3d-1cc2-476d-8e80-ad57c47304fa None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:01.574489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:01.575012 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.575289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.575563 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.576404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:03:01.576708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:01.578361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:01.587996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:01.592459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1189/2373] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:03:01.597910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dcdb3-2082-4821-9417-36a899e371f0 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:01.612065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dcdb3-2082-4821-9417-36a899e371f0 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:01.618884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287c06c-2ff3-4745-ad5e-b4f19c90fa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:01.619548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287c06c-2ff3-4745-ad5e-b4f19c90fa40 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.619849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287c06c-2ff3-4745-ad5e-b4f19c90fa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.620150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287c06c-2ff3-4745-ad5e-b4f19c90fa40 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.620851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287c06c-2ff3-4745-ad5e-b4f19c90fa40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:01.622877 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287c06c-2ff3-4745-ad5e-b4f19c90fa40 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:01.654009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1185/2374] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:03:01 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:01.655095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:01.665474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0661c-2a89-4b70-a4d0-8fe3ee97f743 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:01.723399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0d6dd-fe52-490d-8656-7779fde39dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:01.724078 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d6dd-fe52-490d-8656-7779fde39dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.724359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d6dd-fe52-490d-8656-7779fde39dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.724636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d6dd-fe52-490d-8656-7779fde39dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.725383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0d6dd-fe52-490d-8656-7779fde39dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:01.727553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0d6dd-fe52-490d-8656-7779fde39dc8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:01.748568 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1190/2375] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:03:01 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:01.752660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287c06c-2ff3-4745-ad5e-b4f19c90fa40 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:01.763023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287c06c-2ff3-4745-ad5e-b4f19c90fa40 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:01.767927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084da1dc-4884-47f8-bac9-03b989903c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:01.768588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084da1dc-4884-47f8-bac9-03b989903c2e admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.768873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084da1dc-4884-47f8-bac9-03b989903c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.769149 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084da1dc-4884-47f8-bac9-03b989903c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.769947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084da1dc-4884-47f8-bac9-03b989903c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:01.771563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084da1dc-4884-47f8-bac9-03b989903c2e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:01.782362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1186/2376] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:01.786997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0d6dd-fe52-490d-8656-7779fde39dc8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:01.803031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0d6dd-fe52-490d-8656-7779fde39dc8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:01.808598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbf30d-f5d3-400e-8977-b606e7fce102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:01.809534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbf30d-f5d3-400e-8977-b606e7fce102 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.809908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbf30d-f5d3-400e-8977-b606e7fce102 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.810161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbf30d-f5d3-400e-8977-b606e7fce102 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.810946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbf30d-f5d3-400e-8977-b606e7fce102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:01.812541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbf30d-f5d3-400e-8977-b606e7fce102 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:01.824185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1191/2377] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:01.828844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084da1dc-4884-47f8-bac9-03b989903c2e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:01.836432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:01.840528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084da1dc-4884-47f8-bac9-03b989903c2e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:01.846832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761181a-e51a-480b-b307-23e70cc904e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:01.847777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761181a-e51a-480b-b307-23e70cc904e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.848418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761181a-e51a-480b-b307-23e70cc904e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.848721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761181a-e51a-480b-b307-23e70cc904e5 admin admin] PATH_INFO: `/v3/projects/2d97141e75b14b7cb825789a0756ff01/users/d35b982c88d1490b87d26d08071f22a9/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.864460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761181a-e51a-480b-b307-23e70cc904e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d97141e75b14b7cb825789a0756ff01, user_id=d35b982c88d1490b87d26d08071f22a9, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:01.867587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761181a-e51a-480b-b307-23e70cc904e5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:01.882214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1187/2378] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:01 2022] PUT /identity/v3/projects/2d97141e75b14b7cb825789a0756ff01/users/d35b982c88d1490b87d26d08071f22a9/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:01.893566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa3774-9998-49b0-871f-45ead15d566d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:01.893915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa3774-9998-49b0-871f-45ead15d566d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:01.894200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa3774-9998-49b0-871f-45ead15d566d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:01.923762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-67aa3774-9998-49b0-871f-45ead15d566d None None] MFA Rules not processed for user `d35b982c88d1490b87d26d08071f22a9`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:01.924912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-67aa3774-9998-49b0-871f-45ead15d566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:01.992591 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1192/2379] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:01.997472 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761181a-e51a-480b-b307-23e70cc904e5 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.065145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761181a-e51a-480b-b307-23e70cc904e5 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.071367 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddf467-e83c-43a1-8c2f-5cea950cbd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.072041 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddf467-e83c-43a1-8c2f-5cea950cbd66 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.072376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddf467-e83c-43a1-8c2f-5cea950cbd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.072657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddf467-e83c-43a1-8c2f-5cea950cbd66 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.073372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddf467-e83c-43a1-8c2f-5cea950cbd66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.075449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddf467-e83c-43a1-8c2f-5cea950cbd66 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.105795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1188/2380] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:03:01 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:02.110165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa3774-9998-49b0-871f-45ead15d566d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.121876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa3774-9998-49b0-871f-45ead15d566d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.180310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e41159-17dc-424a-8110-1057010da43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.180922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41159-17dc-424a-8110-1057010da43b admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.181208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41159-17dc-424a-8110-1057010da43b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.181481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e41159-17dc-424a-8110-1057010da43b admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.182206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e41159-17dc-424a-8110-1057010da43b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.184126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e41159-17dc-424a-8110-1057010da43b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.202392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:02.209383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1193/2381] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:03:02 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:02.213536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddf467-e83c-43a1-8c2f-5cea950cbd66 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.224231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddf467-e83c-43a1-8c2f-5cea950cbd66 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.228980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f107-de03-489f-8e5d-1c7530a7e546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.229663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f107-de03-489f-8e5d-1c7530a7e546 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.229972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f107-de03-489f-8e5d-1c7530a7e546 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.230257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5f107-de03-489f-8e5d-1c7530a7e546 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.231057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5f107-de03-489f-8e5d-1c7530a7e546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.232658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5f107-de03-489f-8e5d-1c7530a7e546 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.245152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1189/2382] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:02.249396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e41159-17dc-424a-8110-1057010da43b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.261126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e41159-17dc-424a-8110-1057010da43b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.267880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208511d4-77a9-4f07-8047-51f42b85d231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.268556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208511d4-77a9-4f07-8047-51f42b85d231 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.268897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208511d4-77a9-4f07-8047-51f42b85d231 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.269175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208511d4-77a9-4f07-8047-51f42b85d231 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.269956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208511d4-77a9-4f07-8047-51f42b85d231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.271633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208511d4-77a9-4f07-8047-51f42b85d231 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.285322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1194/2383] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:02.290020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f107-de03-489f-8e5d-1c7530a7e546 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.300873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5f107-de03-489f-8e5d-1c7530a7e546 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.306580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f7d41-3543-4165-8587-c4e9b359f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.307677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f7d41-3543-4165-8587-c4e9b359f79d admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.307958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f7d41-3543-4165-8587-c4e9b359f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.308229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f7d41-3543-4165-8587-c4e9b359f79d admin admin] PATH_INFO: `/v3/projects/2c3c9793d2b84770acae86127543e6b7/users/e31ed4dc5477433c9242f8b6bdd42cef/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.325310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f7d41-3543-4165-8587-c4e9b359f79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c3c9793d2b84770acae86127543e6b7, user_id=e31ed4dc5477433c9242f8b6bdd42cef, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.328060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f7d41-3543-4165-8587-c4e9b359f79d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.343628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1190/2384] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:02 2022] PUT /identity/v3/projects/2c3c9793d2b84770acae86127543e6b7/users/e31ed4dc5477433c9242f8b6bdd42cef/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:02.354472 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f83be3-2600-4bac-9063-6764f813e1be None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.354796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f83be3-2600-4bac-9063-6764f813e1be None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.355095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f83be3-2600-4bac-9063-6764f813e1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.387406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-f3f83be3-2600-4bac-9063-6764f813e1be None None] MFA Rules not processed for user `e31ed4dc5477433c9242f8b6bdd42cef`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:02.389156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f3f83be3-2600-4bac-9063-6764f813e1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:02.464832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1195/2385] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:02.469467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f7d41-3543-4165-8587-c4e9b359f79d admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.543583 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f7d41-3543-4165-8587-c4e9b359f79d admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.552652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b583-50a1-4941-b5dc-3f971a673969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.553817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b583-50a1-4941-b5dc-3f971a673969 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.554416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b583-50a1-4941-b5dc-3f971a673969 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.554782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b583-50a1-4941-b5dc-3f971a673969 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.555480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67b583-50a1-4941-b5dc-3f971a673969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.557409 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67b583-50a1-4941-b5dc-3f971a673969 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.588896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1191/2386] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:03:02 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:02.594008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f83be3-2600-4bac-9063-6764f813e1be None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.605337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f83be3-2600-4bac-9063-6764f813e1be None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.668634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2d27-7ce3-4f49-8372-199d325f5361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.669321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2d27-7ce3-4f49-8372-199d325f5361 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.669599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2d27-7ce3-4f49-8372-199d325f5361 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.669899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df2d27-7ce3-4f49-8372-199d325f5361 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.671062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df2d27-7ce3-4f49-8372-199d325f5361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.674294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df2d27-7ce3-4f49-8372-199d325f5361 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.703706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1196/2387] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:03:02 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:02.707719 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b583-50a1-4941-b5dc-3f971a673969 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.719295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b583-50a1-4941-b5dc-3f971a673969 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.724835 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc06711-1ff1-4e20-b48e-48b7e32c65c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.725656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc06711-1ff1-4e20-b48e-48b7e32c65c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.726105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc06711-1ff1-4e20-b48e-48b7e32c65c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.726395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc06711-1ff1-4e20-b48e-48b7e32c65c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.727127 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc06711-1ff1-4e20-b48e-48b7e32c65c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.728751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc06711-1ff1-4e20-b48e-48b7e32c65c7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.740228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1192/2388] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:03:02.744920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2d27-7ce3-4f49-8372-199d325f5361 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.755107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df2d27-7ce3-4f49-8372-199d325f5361 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.760327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ad353-0ce5-4ee9-8d51-ab3aeb74e183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.761227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ad353-0ce5-4ee9-8d51-ab3aeb74e183 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.761510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ad353-0ce5-4ee9-8d51-ab3aeb74e183 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.761856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ad353-0ce5-4ee9-8d51-ab3aeb74e183 admin admin] PATH_INFO: `/v3/projects/644d82b874c44b92bcefa81ef4ab7f6b/users/ace1d6883bfc44aaaaf6e817ff25f46d/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.779933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ad353-0ce5-4ee9-8d51-ab3aeb74e183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644d82b874c44b92bcefa81ef4ab7f6b, user_id=ace1d6883bfc44aaaaf6e817ff25f46d, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.782518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ad353-0ce5-4ee9-8d51-ab3aeb74e183 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.798334 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1197/2389] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:02 2022] PUT /identity/v3/projects/644d82b874c44b92bcefa81ef4ab7f6b/users/ace1d6883bfc44aaaaf6e817ff25f46d/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:02.802475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc06711-1ff1-4e20-b48e-48b7e32c65c7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.809186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:02.813156 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc06711-1ff1-4e20-b48e-48b7e32c65c7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.877728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e8fed-0821-4864-a1c4-fe5ca4a082c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.878443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8fed-0821-4864-a1c4-fe5ca4a082c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.878730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8fed-0821-4864-a1c4-fe5ca4a082c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.878994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e8fed-0821-4864-a1c4-fe5ca4a082c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.879730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e8fed-0821-4864-a1c4-fe5ca4a082c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.881329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e8fed-0821-4864-a1c4-fe5ca4a082c6 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.892796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1193/2390] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:03:02.897249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ad353-0ce5-4ee9-8d51-ab3aeb74e183 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:02.910373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ad353-0ce5-4ee9-8d51-ab3aeb74e183 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:02.916826 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:02.917975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.918359 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.918939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] PATH_INFO: `/v3/projects/644d82b874c44b92bcefa81ef4ab7f6b/users/ace1d6883bfc44aaaaf6e817ff25f46d/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.921919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644d82b874c44b92bcefa81ef4ab7f6b, user_id=ace1d6883bfc44aaaaf6e817ff25f46d, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:02.924807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:02.937575 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1198/2391] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:02 2022] PUT /identity/v3/projects/644d82b874c44b92bcefa81ef4ab7f6b/users/ace1d6883bfc44aaaaf6e817ff25f46d/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:02.949182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e055f-7b8a-42c6-babc-ecaa86ae5d77 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:02.949343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e055f-7b8a-42c6-babc-ecaa86ae5d77 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:02.949621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e055f-7b8a-42c6-babc-ecaa86ae5d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:02.979474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f96e055f-7b8a-42c6-babc-ecaa86ae5d77 None None] MFA Rules not processed for user `ace1d6883bfc44aaaaf6e817ff25f46d`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:02.980855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f96e055f-7b8a-42c6-babc-ecaa86ae5d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:03.071922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1194/2392] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:03:03.087425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:03.089336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:03.125227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:03.152856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4d40d-4678-4bbe-be98-d8acbe7c50f0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:03.156769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde53b95-59d2-45d9-beb6-3a340351af91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:03.157427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde53b95-59d2-45d9-beb6-3a340351af91 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:03.157705 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde53b95-59d2-45d9-beb6-3a340351af91 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:03.158007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde53b95-59d2-45d9-beb6-3a340351af91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:03.168242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde53b95-59d2-45d9-beb6-3a340351af91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:03.169730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde53b95-59d2-45d9-beb6-3a340351af91 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:03.182190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1199/2393] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:03.225428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e055f-7b8a-42c6-babc-ecaa86ae5d77 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:03.227295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f96e055f-7b8a-42c6-babc-ecaa86ae5d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:03.240318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e055f-7b8a-42c6-babc-ecaa86ae5d77 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:03.245385 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2e522-27bd-4aea-b2fc-e8537c650bd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:03.246046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2e522-27bd-4aea-b2fc-e8537c650bd7 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:03.246046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2e522-27bd-4aea-b2fc-e8537c650bd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:03.246205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2e522-27bd-4aea-b2fc-e8537c650bd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:03.310093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2e522-27bd-4aea-b2fc-e8537c650bd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:03.311641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2e522-27bd-4aea-b2fc-e8537c650bd7 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:03.321953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1195/2394] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:06.290708 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde53b95-59d2-45d9-beb6-3a340351af91 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:06.358581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde53b95-59d2-45d9-beb6-3a340351af91 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:06.363871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:06.364819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:06.365081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:06.365408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] PATH_INFO: `/v3/users/d35b982c88d1490b87d26d08071f22a9` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:06.367726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35b982c88d1490b87d26d08071f22a9)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:06.370313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:06.423499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35b982c88d1490b87d26d08071f22a9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:06.436552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35b982c88d1490b87d26d08071f22a9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:06.446205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35b982c88d1490b87d26d08071f22a9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:06.447239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35b982c88d1490b87d26d08071f22a9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:06.454874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35b982c88d1490b87d26d08071f22a9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:06.461313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1200/2395] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:03:06 2022] DELETE /identity/v3/users/d35b982c88d1490b87d26d08071f22a9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:06.466493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6bb62-09f9-4764-bda3-b2bb60634be5 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:06.466997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6bb62-09f9-4764-bda3-b2bb60634be5 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:06.467094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6bb62-09f9-4764-bda3-b2bb60634be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:06.485173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e7d6bb62-09f9-4764-bda3-b2bb60634be5 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:06.486466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e7d6bb62-09f9-4764-bda3-b2bb60634be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:06.506817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:06.551215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1196/2396] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:03:06.561720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:06.563506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:06.624476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931130ab-8b5b-4409-aa0d-4cf0ab4a0c3a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:06.628489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8e2f-949c-4eaf-99a9-9a03bf67c814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:06.629131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8e2f-949c-4eaf-99a9-9a03bf67c814 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:06.629417 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8e2f-949c-4eaf-99a9-9a03bf67c814 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:06.629695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8e2f-949c-4eaf-99a9-9a03bf67c814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:06.636950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:06.641415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb8e2f-949c-4eaf-99a9-9a03bf67c814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:06.643012 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb8e2f-949c-4eaf-99a9-9a03bf67c814 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:06.654485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1201/2397] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:06.949414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6bb62-09f9-4764-bda3-b2bb60634be5 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:06.951013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e7d6bb62-09f9-4764-bda3-b2bb60634be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:06.964612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6bb62-09f9-4764-bda3-b2bb60634be5 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:06.969515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:06.970365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:06.970644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:06.970904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] PATH_INFO: `/v3/projects/2d97141e75b14b7cb825789a0756ff01` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:06.972109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d97141e75b14b7cb825789a0756ff01)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:06.974050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:07.024362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d97141e75b14b7cb825789a0756ff01', 'request_id': 'req-d0f3d348-a25b-4525-8597-50147909b484'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.031254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d97141e75b14b7cb825789a0756ff01', 'request_id': 'req-d0f3d348-a25b-4525-8597-50147909b484'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.041521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d97141e75b14b7cb825789a0756ff01', 'request_id': 'req-d0f3d348-a25b-4525-8597-50147909b484'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.057935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] The token cache is being invalidate because project 2d97141e75b14b7cb825789a0756ff01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:07.058227 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.058930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.060642 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1197/2398] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:06 2022] DELETE /identity/v3/projects/2d97141e75b14b7cb825789a0756ff01 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:07.066060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8e2f-949c-4eaf-99a9-9a03bf67c814 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:07.076543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8e2f-949c-4eaf-99a9-9a03bf67c814 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:07.140401 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:07.141299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:07.141559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:07.141871 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] PATH_INFO: `/v3/users/e31ed4dc5477433c9242f8b6bdd42cef` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:07.143103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e31ed4dc5477433c9242f8b6bdd42cef)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:07.145080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:07.197370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e31ed4dc5477433c9242f8b6bdd42cef'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.211732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e31ed4dc5477433c9242f8b6bdd42cef'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.221460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e31ed4dc5477433c9242f8b6bdd42cef'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.222288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e31ed4dc5477433c9242f8b6bdd42cef'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.230179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e31ed4dc5477433c9242f8b6bdd42cef'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.236483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1202/2399] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:03:07 2022] DELETE /identity/v3/users/e31ed4dc5477433c9242f8b6bdd42cef => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:07.534880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:07.536572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:07.551458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3d348-a25b-4525-8597-50147909b484 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:07.615200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:07.616145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:07.616511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:07.616860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] PATH_INFO: `/v3/projects/2c3c9793d2b84770acae86127543e6b7` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:07.618341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c3c9793d2b84770acae86127543e6b7)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:07.620548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:07.681432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c3c9793d2b84770acae86127543e6b7', 'request_id': 'req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.682544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:07.690405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c3c9793d2b84770acae86127543e6b7', 'request_id': 'req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.700735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c3c9793d2b84770acae86127543e6b7', 'request_id': 'req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.712739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] The token cache is being invalidate because project 2c3c9793d2b84770acae86127543e6b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:07.713030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.713673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.715549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1198/2400] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:07 2022] DELETE /identity/v3/projects/2c3c9793d2b84770acae86127543e6b7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:07.719562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:07.721011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:07.734145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731087-a8c0-4b37-84c0-11408e54ae08 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:07.794090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:07.794962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:07.795242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:07.795505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] PATH_INFO: `/v3/users/ace1d6883bfc44aaaaf6e817ff25f46d` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:07.796777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace1d6883bfc44aaaaf6e817ff25f46d)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:07.798822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:07.812782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:07.855106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace1d6883bfc44aaaaf6e817ff25f46d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.867228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace1d6883bfc44aaaaf6e817ff25f46d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.877130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace1d6883bfc44aaaaf6e817ff25f46d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.877818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace1d6883bfc44aaaaf6e817ff25f46d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.886062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5f8cce03-f03f-434f-9d6e-2e39d420ec86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace1d6883bfc44aaaaf6e817ff25f46d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:07.892619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1203/2401] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:03:07 2022] DELETE /identity/v3/users/ace1d6883bfc44aaaaf6e817ff25f46d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:08.268206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:08.269676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:08.284052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e2cc9-ee23-49fb-9f32-286c304ad9c5 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:08.353740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:08.354646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.354929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.355206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] PATH_INFO: `/v3/projects/644d82b874c44b92bcefa81ef4ab7f6b` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.356477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=644d82b874c44b92bcefa81ef4ab7f6b)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:08.358460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:08.408391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644d82b874c44b92bcefa81ef4ab7f6b', 'request_id': 'req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:08.415064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644d82b874c44b92bcefa81ef4ab7f6b', 'request_id': 'req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:08.423680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644d82b874c44b92bcefa81ef4ab7f6b', 'request_id': 'req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:08.437386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] The token cache is being invalidate because project 644d82b874c44b92bcefa81ef4ab7f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:08.439868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:08.442766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-5fb42e27-3aff-44fe-afcf-09ef8c1002a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:08.444403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1199/2402] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:08 2022] DELETE /identity/v3/projects/644d82b874c44b92bcefa81ef4ab7f6b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:08.451663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd716-1fef-46ea-a4d3-a9da64abe11e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.452058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd716-1fef-46ea-a4d3-a9da64abe11e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.452445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd716-1fef-46ea-a4d3-a9da64abe11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.475101 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b7ecd716-1fef-46ea-a4d3-a9da64abe11e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:08.476307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b7ecd716-1fef-46ea-a4d3-a9da64abe11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:08.552132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1204/2403] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:08.563222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c69f5-5d2b-47de-a732-c8aaf02c2394 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.563773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c69f5-5d2b-47de-a732-c8aaf02c2394 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.563773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c69f5-5d2b-47de-a732-c8aaf02c2394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.584304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-654c69f5-5d2b-47de-a732-c8aaf02c2394 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:08.585509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-654c69f5-5d2b-47de-a732-c8aaf02c2394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:08.595131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1200/2404] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:08.599590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd716-1fef-46ea-a4d3-a9da64abe11e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:08.610532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd716-1fef-46ea-a4d3-a9da64abe11e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:08.618512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:08.619479 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.619880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.620277 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.621368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:03:08.621752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:08.624095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:08.640385 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1205/2405] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:03:08.647252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c69f5-5d2b-47de-a732-c8aaf02c2394 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:08.660037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c69f5-5d2b-47de-a732-c8aaf02c2394 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:08.666983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2573b-e3fa-4fce-81aa-de0cd28fe082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:08.667651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2573b-e3fa-4fce-81aa-de0cd28fe082 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.667932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2573b-e3fa-4fce-81aa-de0cd28fe082 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.668204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2573b-e3fa-4fce-81aa-de0cd28fe082 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.668900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2573b-e3fa-4fce-81aa-de0cd28fe082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:08.670886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2573b-e3fa-4fce-81aa-de0cd28fe082 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:08.701841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1201/2406] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:03:08 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:08.706302 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:08.716614 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620747fa-fdb2-4f0d-94ae-657652043703 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:08.774255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef950653-99ee-4c04-b1a1-0e241e8629e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:08.774976 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef950653-99ee-4c04-b1a1-0e241e8629e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.775261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef950653-99ee-4c04-b1a1-0e241e8629e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.775519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef950653-99ee-4c04-b1a1-0e241e8629e8 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.776268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef950653-99ee-4c04-b1a1-0e241e8629e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:08.778734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef950653-99ee-4c04-b1a1-0e241e8629e8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:08.806351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1206/2407] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:03:08 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:08.810953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2573b-e3fa-4fce-81aa-de0cd28fe082 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:08.825055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2573b-e3fa-4fce-81aa-de0cd28fe082 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:08.830080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304008-b709-4088-b396-2ccff5d9a4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:08.830745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304008-b709-4088-b396-2ccff5d9a4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.831019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304008-b709-4088-b396-2ccff5d9a4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.831294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304008-b709-4088-b396-2ccff5d9a4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.832023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c304008-b709-4088-b396-2ccff5d9a4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:08.833622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c304008-b709-4088-b396-2ccff5d9a4c7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:08.844912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1202/2408] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:08.849021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef950653-99ee-4c04-b1a1-0e241e8629e8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:08.859742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef950653-99ee-4c04-b1a1-0e241e8629e8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:08.864733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc5c5e-d9e3-4c78-8ff7-66d6d1bc490b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:08.865388 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc5c5e-d9e3-4c78-8ff7-66d6d1bc490b admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.865680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc5c5e-d9e3-4c78-8ff7-66d6d1bc490b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.865980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc5c5e-d9e3-4c78-8ff7-66d6d1bc490b admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.866753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc5c5e-d9e3-4c78-8ff7-66d6d1bc490b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:08.868947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc5c5e-d9e3-4c78-8ff7-66d6d1bc490b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:08.871106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:08.882637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1207/2409] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:08.887494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304008-b709-4088-b396-2ccff5d9a4c7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:08.897498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304008-b709-4088-b396-2ccff5d9a4c7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:08.902878 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:08.903846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.904162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.904436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] PATH_INFO: `/v3/projects/7cb06b42dc4f4a968fa31e06364d8759/users/921d43ebb6f54a1eb80a6888e8d50073/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.919785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb06b42dc4f4a968fa31e06364d8759, user_id=921d43ebb6f54a1eb80a6888e8d50073, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:08.922510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:08.937352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1203/2410] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:08 2022] PUT /identity/v3/projects/7cb06b42dc4f4a968fa31e06364d8759/users/921d43ebb6f54a1eb80a6888e8d50073/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:08.950520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54568e5-ff1d-46b8-8727-7f9739d770c2 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:08.951005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54568e5-ff1d-46b8-8727-7f9739d770c2 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:08.951371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54568e5-ff1d-46b8-8727-7f9739d770c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:08.981944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b54568e5-ff1d-46b8-8727-7f9739d770c2 None None] MFA Rules not processed for user `921d43ebb6f54a1eb80a6888e8d50073`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:08.983185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b54568e5-ff1d-46b8-8727-7f9739d770c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:09.050581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1208/2411] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:09.063141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:09.065364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:09.073821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:09.132711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5f1b4-99da-48b5-a637-84f51cf0cced admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:09.137419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded8c87-d400-4c4a-a757-2c74791e9b1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:09.138171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded8c87-d400-4c4a-a757-2c74791e9b1d service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:09.138531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded8c87-d400-4c4a-a757-2c74791e9b1d service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:09.138811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded8c87-d400-4c4a-a757-2c74791e9b1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:09.148580 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded8c87-d400-4c4a-a757-2c74791e9b1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:09.150304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded8c87-d400-4c4a-a757-2c74791e9b1d service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:09.161427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1204/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:09.179883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54568e5-ff1d-46b8-8727-7f9739d770c2 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:09.261703 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54568e5-ff1d-46b8-8727-7f9739d770c2 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:09.266466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:09.267657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:09.267657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:09.267901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] PATH_INFO: `/v3/users/921d43ebb6f54a1eb80a6888e8d50073` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:09.269506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=921d43ebb6f54a1eb80a6888e8d50073)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:09.271521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:09.327483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921d43ebb6f54a1eb80a6888e8d50073'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.341153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921d43ebb6f54a1eb80a6888e8d50073'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.348603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921d43ebb6f54a1eb80a6888e8d50073'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.349211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921d43ebb6f54a1eb80a6888e8d50073'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.356329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921d43ebb6f54a1eb80a6888e8d50073'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.362272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1209/2413] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:03:09 2022] DELETE /identity/v3/users/921d43ebb6f54a1eb80a6888e8d50073 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:09.366817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae675d9-da79-49dd-aa69-e64634658193 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:09.367134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae675d9-da79-49dd-aa69-e64634658193 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:09.367418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae675d9-da79-49dd-aa69-e64634658193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:09.386086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2ae675d9-da79-49dd-aa69-e64634658193 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:09.387306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2ae675d9-da79-49dd-aa69-e64634658193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:09.453056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1205/2414] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:03:09.462927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:09.464358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:09.474328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:09.526981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e2f3-d044-4d6a-8c0e-d9c1c8ff8e26 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:09.530877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e283bb-11ec-4bab-ace3-ca22f1633d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:09.531589 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e283bb-11ec-4bab-ace3-ca22f1633d0e service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:09.531879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e283bb-11ec-4bab-ace3-ca22f1633d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:09.532185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e283bb-11ec-4bab-ace3-ca22f1633d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:09.542356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e283bb-11ec-4bab-ace3-ca22f1633d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:09.543781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e283bb-11ec-4bab-ace3-ca22f1633d0e service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:09.554170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1210/2415] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:09.836098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae675d9-da79-49dd-aa69-e64634658193 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:09.838362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2ae675d9-da79-49dd-aa69-e64634658193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:09.850523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae675d9-da79-49dd-aa69-e64634658193 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:09.859060 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:09.859978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:09.860245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:09.860523 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] PATH_INFO: `/v3/projects/7cb06b42dc4f4a968fa31e06364d8759` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:09.861828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb06b42dc4f4a968fa31e06364d8759)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:09.863725 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:09.916813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb06b42dc4f4a968fa31e06364d8759', 'request_id': 'req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.923544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb06b42dc4f4a968fa31e06364d8759', 'request_id': 'req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.933006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb06b42dc4f4a968fa31e06364d8759', 'request_id': 'req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.944051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] The token cache is being invalidate because project 7cb06b42dc4f4a968fa31e06364d8759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:09.944337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.944977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-cbd81d03-d5a1-4ea7-9d85-3ededdaad943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:09.947193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1206/2416] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:09 2022] DELETE /identity/v3/projects/7cb06b42dc4f4a968fa31e06364d8759 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:09.952797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128fac9-e74e-4f93-8dba-6787553b6dd6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:09.953293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128fac9-e74e-4f93-8dba-6787553b6dd6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:09.953397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128fac9-e74e-4f93-8dba-6787553b6dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:09.972471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-4128fac9-e74e-4f93-8dba-6787553b6dd6 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:09.973688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4128fac9-e74e-4f93-8dba-6787553b6dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:10.055546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1211/2417] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:10.063749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273bb6-709c-45b3-b9a2-b3cea81bccfd None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.064109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273bb6-709c-45b3-b9a2-b3cea81bccfd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.064423 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273bb6-709c-45b3-b9a2-b3cea81bccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.085508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-be273bb6-709c-45b3-b9a2-b3cea81bccfd None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:10.086848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-be273bb6-709c-45b3-b9a2-b3cea81bccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:10.096100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1207/2418] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:10.100704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128fac9-e74e-4f93-8dba-6787553b6dd6 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.111605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128fac9-e74e-4f93-8dba-6787553b6dd6 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.117860 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.118553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.118856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.119116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.119850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:03:10.120160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.121813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.135183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1212/2419] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:03:10.141427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be273bb6-709c-45b3-b9a2-b3cea81bccfd None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.155615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be273bb6-709c-45b3-b9a2-b3cea81bccfd None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.164811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e50a14-5b2a-4750-8bec-3b57fc709380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.166031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50a14-5b2a-4750-8bec-3b57fc709380 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.166307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50a14-5b2a-4750-8bec-3b57fc709380 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.166573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50a14-5b2a-4750-8bec-3b57fc709380 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.167265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e50a14-5b2a-4750-8bec-3b57fc709380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.169173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e50a14-5b2a-4750-8bec-3b57fc709380 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.199246 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1208/2420] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:03:10 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:10.204177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.219395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759399b9-a1b0-4327-a9e2-921941a3719e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.292759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689351a-749d-4dfb-acd4-b2a2a933991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.293433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689351a-749d-4dfb-acd4-b2a2a933991c admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.293722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689351a-749d-4dfb-acd4-b2a2a933991c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.294036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689351a-749d-4dfb-acd4-b2a2a933991c admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.294752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689351a-749d-4dfb-acd4-b2a2a933991c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.297368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689351a-749d-4dfb-acd4-b2a2a933991c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.321719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1213/2421] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:03:10 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:03:10.326359 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e50a14-5b2a-4750-8bec-3b57fc709380 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.334797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:10.342180 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e50a14-5b2a-4750-8bec-3b57fc709380 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.347473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00b9a7-026c-4b3e-ab2d-fab1a15f21c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.348205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b9a7-026c-4b3e-ab2d-fab1a15f21c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.348496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b9a7-026c-4b3e-ab2d-fab1a15f21c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.348787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00b9a7-026c-4b3e-ab2d-fab1a15f21c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.349521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00b9a7-026c-4b3e-ab2d-fab1a15f21c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.351207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00b9a7-026c-4b3e-ab2d-fab1a15f21c1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.362674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1209/2422] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:10.366632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689351a-749d-4dfb-acd4-b2a2a933991c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.377216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689351a-749d-4dfb-acd4-b2a2a933991c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.382136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f98b2f-dabe-4869-b908-593a0740d154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.382792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98b2f-dabe-4869-b908-593a0740d154 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.383079 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98b2f-dabe-4869-b908-593a0740d154 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.383357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f98b2f-dabe-4869-b908-593a0740d154 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.384214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98b2f-dabe-4869-b908-593a0740d154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.385857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f98b2f-dabe-4869-b908-593a0740d154 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.396968 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1214/2423] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:10.401259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00b9a7-026c-4b3e-ab2d-fab1a15f21c1 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.411979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00b9a7-026c-4b3e-ab2d-fab1a15f21c1 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.417796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22754a51-2564-4238-8658-bc55851c183a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.418706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22754a51-2564-4238-8658-bc55851c183a admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.418967 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22754a51-2564-4238-8658-bc55851c183a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.419235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22754a51-2564-4238-8658-bc55851c183a admin admin] PATH_INFO: `/v3/projects/75744826c478471394678011f538282f/users/1ec135066eef4f73a690780ec94920ed/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.435994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22754a51-2564-4238-8658-bc55851c183a admin admin] RBAC: Authorizing `identity:create_grant(project_id=75744826c478471394678011f538282f, user_id=1ec135066eef4f73a690780ec94920ed, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.438702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22754a51-2564-4238-8658-bc55851c183a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.455680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1210/2424] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:10 2022] PUT /identity/v3/projects/75744826c478471394678011f538282f/users/1ec135066eef4f73a690780ec94920ed/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:10.466972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64cd99-9a81-4f4d-95c3-c8e724aef66f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.467274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64cd99-9a81-4f4d-95c3-c8e724aef66f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.467560 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64cd99-9a81-4f4d-95c3-c8e724aef66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.495235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5c64cd99-9a81-4f4d-95c3-c8e724aef66f None None] MFA Rules not processed for user `1ec135066eef4f73a690780ec94920ed`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:10.496453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5c64cd99-9a81-4f4d-95c3-c8e724aef66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:10.500680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:10.563722 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1215/2425] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:03:10.570274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22754a51-2564-4238-8658-bc55851c183a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.641473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22754a51-2564-4238-8658-bc55851c183a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.646418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12081e1a-c1ee-445a-8c8f-20f61a271d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.647075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12081e1a-c1ee-445a-8c8f-20f61a271d53 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.647382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12081e1a-c1ee-445a-8c8f-20f61a271d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.647653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12081e1a-c1ee-445a-8c8f-20f61a271d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.648352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12081e1a-c1ee-445a-8c8f-20f61a271d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.650306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12081e1a-c1ee-445a-8c8f-20f61a271d53 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.680337 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1211/2426] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:03:10 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:03:10.685047 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64cd99-9a81-4f4d-95c3-c8e724aef66f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.696721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64cd99-9a81-4f4d-95c3-c8e724aef66f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.755197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739cb12-ebdc-4db8-96db-5a1f57ee80d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.755868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739cb12-ebdc-4db8-96db-5a1f57ee80d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.756151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739cb12-ebdc-4db8-96db-5a1f57ee80d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.756422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739cb12-ebdc-4db8-96db-5a1f57ee80d8 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.757083 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739cb12-ebdc-4db8-96db-5a1f57ee80d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.759294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739cb12-ebdc-4db8-96db-5a1f57ee80d8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.790943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1216/2427] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:03:10 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:10.792444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12081e1a-c1ee-445a-8c8f-20f61a271d53 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.803066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12081e1a-c1ee-445a-8c8f-20f61a271d53 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.808816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8868d-9a93-48a3-b70c-df2c58bcd0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.809480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8868d-9a93-48a3-b70c-df2c58bcd0cf admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.809751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8868d-9a93-48a3-b70c-df2c58bcd0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.810075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8868d-9a93-48a3-b70c-df2c58bcd0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.810781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8868d-9a93-48a3-b70c-df2c58bcd0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.812394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8868d-9a93-48a3-b70c-df2c58bcd0cf admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.827845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1212/2428] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:10.828720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739cb12-ebdc-4db8-96db-5a1f57ee80d8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.842179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739cb12-ebdc-4db8-96db-5a1f57ee80d8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.850361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798de47-4f14-44ce-9b8b-7382f4d0e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.851354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798de47-4f14-44ce-9b8b-7382f4d0e51c admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.851733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798de47-4f14-44ce-9b8b-7382f4d0e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.852091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798de47-4f14-44ce-9b8b-7382f4d0e51c admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.853470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798de47-4f14-44ce-9b8b-7382f4d0e51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.855123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798de47-4f14-44ce-9b8b-7382f4d0e51c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.865583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1217/2429] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:10.870482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8868d-9a93-48a3-b70c-df2c58bcd0cf admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:10.881751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8868d-9a93-48a3-b70c-df2c58bcd0cf admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:10.888128 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928aa15-1434-40d1-9801-1be939384983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:10.889013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928aa15-1434-40d1-9801-1be939384983 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.889290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928aa15-1434-40d1-9801-1be939384983 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.889568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928aa15-1434-40d1-9801-1be939384983 admin admin] PATH_INFO: `/v3/projects/cbbe8364a14b4eb9a63be992ecc6dd31/users/bec5470bafc34974a5f52f21ea4e5a73/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.907533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928aa15-1434-40d1-9801-1be939384983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbe8364a14b4eb9a63be992ecc6dd31, user_id=bec5470bafc34974a5f52f21ea4e5a73, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:10.912136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928aa15-1434-40d1-9801-1be939384983 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:10.928184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1213/2430] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:10 2022] PUT /identity/v3/projects/cbbe8364a14b4eb9a63be992ecc6dd31/users/bec5470bafc34974a5f52f21ea4e5a73/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:10.938882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0445-2bd6-4ff3-a462-78ceab06b4d9 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:10.939497 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0445-2bd6-4ff3-a462-78ceab06b4d9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:10.939796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df0445-2bd6-4ff3-a462-78ceab06b4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:10.969691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d7df0445-2bd6-4ff3-a462-78ceab06b4d9 None None] MFA Rules not processed for user `bec5470bafc34974a5f52f21ea4e5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:10.970958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d7df0445-2bd6-4ff3-a462-78ceab06b4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:11.046298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1218/2431] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:11.050882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928aa15-1434-40d1-9801-1be939384983 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:11.120361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928aa15-1434-40d1-9801-1be939384983 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:11.126084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643882-d926-44ea-a6f5-23d096b05205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:11.126748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643882-d926-44ea-a6f5-23d096b05205 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:11.127031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643882-d926-44ea-a6f5-23d096b05205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:11.127304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643882-d926-44ea-a6f5-23d096b05205 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:11.127991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25643882-d926-44ea-a6f5-23d096b05205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:11.129951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25643882-d926-44ea-a6f5-23d096b05205 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:11.158529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1214/2432] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:03:11 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:11.162970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df0445-2bd6-4ff3-a462-78ceab06b4d9 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:11.173966 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df0445-2bd6-4ff3-a462-78ceab06b4d9 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:11.184464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:11.236593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216bb7-5e7d-4bbc-bbb2-b48d611e3a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:11.237410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216bb7-5e7d-4bbc-bbb2-b48d611e3a4d admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:11.237963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216bb7-5e7d-4bbc-bbb2-b48d611e3a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:11.237963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e216bb7-5e7d-4bbc-bbb2-b48d611e3a4d admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:11.238683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e216bb7-5e7d-4bbc-bbb2-b48d611e3a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:11.240955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e216bb7-5e7d-4bbc-bbb2-b48d611e3a4d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:11.264418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1219/2433] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:03:11 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:11.268598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643882-d926-44ea-a6f5-23d096b05205 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:11.274397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:11.281284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643882-d926-44ea-a6f5-23d096b05205 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:11.289391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66e1de-988a-4191-9341-bd353f1c4d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:11.290269 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66e1de-988a-4191-9341-bd353f1c4d5a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:11.290554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66e1de-988a-4191-9341-bd353f1c4d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:11.290969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66e1de-988a-4191-9341-bd353f1c4d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:11.291710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66e1de-988a-4191-9341-bd353f1c4d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:11.293387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66e1de-988a-4191-9341-bd353f1c4d5a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:11.304804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1215/2434] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:11.310926 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216bb7-5e7d-4bbc-bbb2-b48d611e3a4d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:11.323787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e216bb7-5e7d-4bbc-bbb2-b48d611e3a4d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:11.329365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a9a88-2411-4367-9d74-8b6ac160373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:11.330584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a9a88-2411-4367-9d74-8b6ac160373f admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:11.330965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a9a88-2411-4367-9d74-8b6ac160373f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:11.331308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a9a88-2411-4367-9d74-8b6ac160373f admin admin] PATH_INFO: `/v3/projects/ae79a80b50d946849dd79d783a590bf1/users/4c7ee76a0edf4556bc8d08983ba9e0c1/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:11.356278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a9a88-2411-4367-9d74-8b6ac160373f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae79a80b50d946849dd79d783a590bf1, user_id=4c7ee76a0edf4556bc8d08983ba9e0c1, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:11.359443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a9a88-2411-4367-9d74-8b6ac160373f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:11.375119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1220/2435] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:11 2022] PUT /identity/v3/projects/ae79a80b50d946849dd79d783a590bf1/users/4c7ee76a0edf4556bc8d08983ba9e0c1/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:11.379299 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66e1de-988a-4191-9341-bd353f1c4d5a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:11.391770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66e1de-988a-4191-9341-bd353f1c4d5a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:11.457542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f282bf-f4a4-45ac-83be-6cae2304e246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:11.458241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f282bf-f4a4-45ac-83be-6cae2304e246 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:11.458533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f282bf-f4a4-45ac-83be-6cae2304e246 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:11.458809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f282bf-f4a4-45ac-83be-6cae2304e246 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:11.459531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f282bf-f4a4-45ac-83be-6cae2304e246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:11.461132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f282bf-f4a4-45ac-83be-6cae2304e246 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:11.471972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1216/2436] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:11.476389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a9a88-2411-4367-9d74-8b6ac160373f admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:11.487310 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a9a88-2411-4367-9d74-8b6ac160373f admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:11.492436 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:11.493951 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:11.494337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:11.494634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] PATH_INFO: `/v3/projects/ae79a80b50d946849dd79d783a590bf1/users/4c7ee76a0edf4556bc8d08983ba9e0c1/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:11.496765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae79a80b50d946849dd79d783a590bf1, user_id=4c7ee76a0edf4556bc8d08983ba9e0c1, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:11.499445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:11.510677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1221/2437] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:11 2022] PUT /identity/v3/projects/ae79a80b50d946849dd79d783a590bf1/users/4c7ee76a0edf4556bc8d08983ba9e0c1/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:11.580098 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca803d5-c4c6-4f93-9212-e7a17d77e846 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:11.580411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca803d5-c4c6-4f93-9212-e7a17d77e846 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:11.580673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca803d5-c4c6-4f93-9212-e7a17d77e846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:11.613247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1ca803d5-c4c6-4f93-9212-e7a17d77e846 None None] MFA Rules not processed for user `4c7ee76a0edf4556bc8d08983ba9e0c1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:11.614530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1ca803d5-c4c6-4f93-9212-e7a17d77e846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:11.698479 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1217/2438] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:11.711832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:11.713687 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:11.778344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f112603-2095-4e0a-9e74-bed48a267d25 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:11.782129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212605c9-23e6-449c-b1bc-4b53b940cf61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:11.782785 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212605c9-23e6-449c-b1bc-4b53b940cf61 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:11.783067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212605c9-23e6-449c-b1bc-4b53b940cf61 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:11.783346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212605c9-23e6-449c-b1bc-4b53b940cf61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:11.841819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212605c9-23e6-449c-b1bc-4b53b940cf61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:11.843309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212605c9-23e6-449c-b1bc-4b53b940cf61 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:11.853081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1222/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:13.386352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca803d5-c4c6-4f93-9212-e7a17d77e846 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:13.396604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca803d5-c4c6-4f93-9212-e7a17d77e846 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:13.401577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:13.402275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:13.402556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:13.402831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:13.471232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:13.472727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:13.482553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1218/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:13.749425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6947ace-4bd6-4afe-958b-c29e2ac6fc07 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:13.750005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6947ace-4bd6-4afe-958b-c29e2ac6fc07 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:13.750005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6947ace-4bd6-4afe-958b-c29e2ac6fc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:13.769156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d6947ace-4bd6-4afe-958b-c29e2ac6fc07 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:13.770459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d6947ace-4bd6-4afe-958b-c29e2ac6fc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:13.785112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:13.847493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1223/2441] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 16:03:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:03:13.895978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:13.897444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:13.913540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:13.998779 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267deb54-0370-46cc-a435-4cd82864baf4 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:14.005820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe7c49-a7bb-4d80-88ee-1e88fad238bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:14.006671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7c49-a7bb-4d80-88ee-1e88fad238bd service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:14.007036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7c49-a7bb-4d80-88ee-1e88fad238bd service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:14.007928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7c49-a7bb-4d80-88ee-1e88fad238bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:14.016911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe7c49-a7bb-4d80-88ee-1e88fad238bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:14.018632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe7c49-a7bb-4d80-88ee-1e88fad238bd service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:14.028525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1219/2442] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 16:03:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 16:03:14.076510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6947ace-4bd6-4afe-958b-c29e2ac6fc07 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:14.092688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6947ace-4bd6-4afe-958b-c29e2ac6fc07 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:14.101074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8461f4a-0774-484f-901d-ce3922c99b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:14.102105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461f4a-0774-484f-901d-ce3922c99b37 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:14.102585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461f4a-0774-484f-901d-ce3922c99b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:14.103324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8461f4a-0774-484f-901d-ce3922c99b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:14.127021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8461f4a-0774-484f-901d-ce3922c99b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:14.128686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8461f4a-0774-484f-901d-ce3922c99b37 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:14.154514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1224/2443] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:18.300418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe7c49-a7bb-4d80-88ee-1e88fad238bd service swift] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:18.480283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe7c49-a7bb-4d80-88ee-1e88fad238bd service swift] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:18.495082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:18.496186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:18.496470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:18.496793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] PATH_INFO: `/v3/users/1ec135066eef4f73a690780ec94920ed` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:18.498503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ec135066eef4f73a690780ec94920ed)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:18.506287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:18.587467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec135066eef4f73a690780ec94920ed'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:18.605937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec135066eef4f73a690780ec94920ed'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:18.616116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec135066eef4f73a690780ec94920ed'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:18.627667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec135066eef4f73a690780ec94920ed'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:18.628588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec135066eef4f73a690780ec94920ed'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:18.635436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1220/2444] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:03:18 2022] DELETE /identity/v3/users/1ec135066eef4f73a690780ec94920ed => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:18.640112 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e312bd-3e72-472e-af9a-0c297010440f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:18.640648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e312bd-3e72-472e-af9a-0c297010440f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:18.640902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e312bd-3e72-472e-af9a-0c297010440f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:18.661095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-23e312bd-3e72-472e-af9a-0c297010440f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:18.662724 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-23e312bd-3e72-472e-af9a-0c297010440f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:18.753418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1225/2445] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:03:18.766196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:18.767533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:18.795165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:18.844727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97519007-671d-41d9-989b-5ee7d4f7c358 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:18.848552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf99947-908b-4484-b903-96ec64f11691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:18.849972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf99947-908b-4484-b903-96ec64f11691 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:18.850362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf99947-908b-4484-b903-96ec64f11691 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:18.850759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf99947-908b-4484-b903-96ec64f11691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:18.863147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf99947-908b-4484-b903-96ec64f11691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:18.865328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf99947-908b-4484-b903-96ec64f11691 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:18.877576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1221/2446] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:03:19.258903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e312bd-3e72-472e-af9a-0c297010440f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:19.260513 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-23e312bd-3e72-472e-af9a-0c297010440f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:19.277329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e312bd-3e72-472e-af9a-0c297010440f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:19.290848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:19.294607 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:19.294896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:19.295174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] PATH_INFO: `/v3/projects/75744826c478471394678011f538282f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:19.296632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=75744826c478471394678011f538282f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:19.298687 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:19.384182 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75744826c478471394678011f538282f', 'request_id': 'req-171a0552-1797-496b-9595-4fab2d716cdb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.394994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75744826c478471394678011f538282f', 'request_id': 'req-171a0552-1797-496b-9595-4fab2d716cdb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.405030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75744826c478471394678011f538282f', 'request_id': 'req-171a0552-1797-496b-9595-4fab2d716cdb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.418311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] The token cache is being invalidate because project 75744826c478471394678011f538282f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:19.418603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.419294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.421225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1226/2447] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:19 2022] DELETE /identity/v3/projects/75744826c478471394678011f538282f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:03:19.426148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf99947-908b-4484-b903-96ec64f11691 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:19.438282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf99947-908b-4484-b903-96ec64f11691 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:19.524076 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:19.525181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:19.525603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:19.526069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] PATH_INFO: `/v3/users/bec5470bafc34974a5f52f21ea4e5a73` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:19.527476 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec5470bafc34974a5f52f21ea4e5a73)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:19.529698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:19.609802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec5470bafc34974a5f52f21ea4e5a73'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.630970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec5470bafc34974a5f52f21ea4e5a73'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.642705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec5470bafc34974a5f52f21ea4e5a73'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.652436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec5470bafc34974a5f52f21ea4e5a73'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.653110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec5470bafc34974a5f52f21ea4e5a73'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:19.661181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1222/2448] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:03:19 2022] DELETE /identity/v3/users/bec5470bafc34974a5f52f21ea4e5a73 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:20.549019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:20.551431 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:20.585381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a0552-1797-496b-9595-4fab2d716cdb admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:20.641939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:20.694428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:20.695847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:20.696358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:20.696744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] PATH_INFO: `/v3/projects/cbbe8364a14b4eb9a63be992ecc6dd31` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:20.698511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbe8364a14b4eb9a63be992ecc6dd31)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:20.701354 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:20.788146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbe8364a14b4eb9a63be992ecc6dd31', 'request_id': 'req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:20.800379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbe8364a14b4eb9a63be992ecc6dd31', 'request_id': 'req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:20.810434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbe8364a14b4eb9a63be992ecc6dd31', 'request_id': 'req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:20.831018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] The token cache is being invalidate because project cbbe8364a14b4eb9a63be992ecc6dd31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:20.831321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:20.832290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:20.834410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1227/2449] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:20 2022] DELETE /identity/v3/projects/cbbe8364a14b4eb9a63be992ecc6dd31 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:20.839037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:20.841649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:20.861700 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6d59f-acfd-4061-af43-2bf7fc494e83 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:20.931894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:20.968349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:20.971030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:20.971711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:20.971992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] PATH_INFO: `/v3/users/4c7ee76a0edf4556bc8d08983ba9e0c1` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:20.974099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7ee76a0edf4556bc8d08983ba9e0c1)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:20.976025 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:21.050023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7ee76a0edf4556bc8d08983ba9e0c1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.072549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7ee76a0edf4556bc8d08983ba9e0c1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.085734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7ee76a0edf4556bc8d08983ba9e0c1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.098971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7ee76a0edf4556bc8d08983ba9e0c1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.099988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-574f3e94-858d-4d77-8040-4ee57cbf8b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7ee76a0edf4556bc8d08983ba9e0c1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.108184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1223/2450] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:03:20 2022] DELETE /identity/v3/users/4c7ee76a0edf4556bc8d08983ba9e0c1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:21.617144 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:21.618696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:21.639240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11a591-efc9-4fb1-9962-5bd7ab8946a0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:21.713288 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:21.714961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:21.715478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:21.716097 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] PATH_INFO: `/v3/projects/ae79a80b50d946849dd79d783a590bf1` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:21.717656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae79a80b50d946849dd79d783a590bf1)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:21.721372 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:21.804080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae79a80b50d946849dd79d783a590bf1', 'request_id': 'req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.828859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae79a80b50d946849dd79d783a590bf1', 'request_id': 'req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.840096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae79a80b50d946849dd79d783a590bf1', 'request_id': 'req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.854699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:21.870223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] The token cache is being invalidate because project ae79a80b50d946849dd79d783a590bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:03:21.870535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.871285 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-22b07aea-5b61-4b2b-99a2-3c0bba757ff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:03:21.876809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1228/2451] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:03:21 2022] DELETE /identity/v3/projects/ae79a80b50d946849dd79d783a590bf1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:21.878313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3c6f-c1eb-4dd0-953b-7a1299a1b21a None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:21.879179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3c6f-c1eb-4dd0-953b-7a1299a1b21a None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:21.879467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c3c6f-c1eb-4dd0-953b-7a1299a1b21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:21.902354 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b40c3c6f-c1eb-4dd0-953b-7a1299a1b21a None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:21.903750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b40c3c6f-c1eb-4dd0-953b-7a1299a1b21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:21.996036 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1224/2452] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:22.008875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715998a0-d755-4918-b775-bdaf818b2d98 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.008875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715998a0-d755-4918-b775-bdaf818b2d98 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.008875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715998a0-d755-4918-b775-bdaf818b2d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.036077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-715998a0-d755-4918-b775-bdaf818b2d98 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:22.037808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-715998a0-d755-4918-b775-bdaf818b2d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:22.053821 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1229/2453] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:22.054257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c3c6f-c1eb-4dd0-953b-7a1299a1b21a None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:22.064401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c3c6f-c1eb-4dd0-953b-7a1299a1b21a None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:22.071730 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:22.072319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.072693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.072883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.073662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:03:22.074000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:22.075957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:22.089825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1225/2454] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:03:22.094132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715998a0-d755-4918-b775-bdaf818b2d98 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:22.104676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715998a0-d755-4918-b775-bdaf818b2d98 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:22.110248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2a657-590d-4b37-adf5-c085449ce91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:22.111175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a657-590d-4b37-adf5-c085449ce91d admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.111396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a657-590d-4b37-adf5-c085449ce91d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.111572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a657-590d-4b37-adf5-c085449ce91d admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.112341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2a657-590d-4b37-adf5-c085449ce91d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:22.114738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2a657-590d-4b37-adf5-c085449ce91d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:22.153130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1230/2455] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:03:22 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:03:22.154639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:22.170676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9aba6-f7da-49b5-a3fc-7cdb4496dfc0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:22.238549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:22.258854 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066979d0-8b03-46d8-9bf3-9da3025b8756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:22.260186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066979d0-8b03-46d8-9bf3-9da3025b8756 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.260671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066979d0-8b03-46d8-9bf3-9da3025b8756 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.261140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066979d0-8b03-46d8-9bf3-9da3025b8756 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.262728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066979d0-8b03-46d8-9bf3-9da3025b8756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:22.265364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066979d0-8b03-46d8-9bf3-9da3025b8756 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:22.298628 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1226/2456] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:03:22 2022] POST /identity/v3/users => generated 285 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:03:22.302747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2a657-590d-4b37-adf5-c085449ce91d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:22.325939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2a657-590d-4b37-adf5-c085449ce91d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:22.341644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5fd79-1b0b-40d6-905c-2511251c9e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:22.342587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5fd79-1b0b-40d6-905c-2511251c9e06 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.343032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5fd79-1b0b-40d6-905c-2511251c9e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.343468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5fd79-1b0b-40d6-905c-2511251c9e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.344586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5fd79-1b0b-40d6-905c-2511251c9e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:22.347014 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5fd79-1b0b-40d6-905c-2511251c9e06 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:22.364088 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1231/2457] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:03:22.368158 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066979d0-8b03-46d8-9bf3-9da3025b8756 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:22.380555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066979d0-8b03-46d8-9bf3-9da3025b8756 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:22.386893 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbccb48-d7e4-4e29-b7dd-10bc23e14537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:22.387913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbccb48-d7e4-4e29-b7dd-10bc23e14537 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.388296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbccb48-d7e4-4e29-b7dd-10bc23e14537 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.388675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbccb48-d7e4-4e29-b7dd-10bc23e14537 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.389570 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbccb48-d7e4-4e29-b7dd-10bc23e14537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:22.392356 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbccb48-d7e4-4e29-b7dd-10bc23e14537 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:22.407595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1227/2458] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:03:22.412114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5fd79-1b0b-40d6-905c-2511251c9e06 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:22.426104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5fd79-1b0b-40d6-905c-2511251c9e06 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:22.431489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:22.432403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.432698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.433018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] PATH_INFO: `/v3/projects/69b62d21e3d54a5592e26594c6483188/users/57bdca8c562a42a3af75f4d7d36bdecc/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.455000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b62d21e3d54a5592e26594c6483188, user_id=57bdca8c562a42a3af75f4d7d36bdecc, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:22.457813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:22.482620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1232/2459] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:03:22 2022] PUT /identity/v3/projects/69b62d21e3d54a5592e26594c6483188/users/57bdca8c562a42a3af75f4d7d36bdecc/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:03:22.488890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997d5ba-6fa9-4ad3-961a-c335ba7fe5c4 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.489717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997d5ba-6fa9-4ad3-961a-c335ba7fe5c4 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.490274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997d5ba-6fa9-4ad3-961a-c335ba7fe5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.515754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a997d5ba-6fa9-4ad3-961a-c335ba7fe5c4 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:22.517654 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a997d5ba-6fa9-4ad3-961a-c335ba7fe5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:22.621929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1228/2460] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:22.629077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:22.631089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:22.714033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384c357-4459-48fb-9e5a-a211649ab562 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:22.719233 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1dde1b-6fa7-47de-9960-34893c5de125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:22.720313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1dde1b-6fa7-47de-9960-34893c5de125 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.720815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1dde1b-6fa7-47de-9960-34893c5de125 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.721518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1dde1b-6fa7-47de-9960-34893c5de125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.736392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1dde1b-6fa7-47de-9960-34893c5de125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:22.738110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1dde1b-6fa7-47de-9960-34893c5de125 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:22.760226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2943381-0d25-44f9-bdb0-43ea1d2931f2 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:22.760552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2943381-0d25-44f9-bdb0-43ea1d2931f2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:22.760829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2943381-0d25-44f9-bdb0-43ea1d2931f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:22.765443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1233/2461] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:22.780666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a2943381-0d25-44f9-bdb0-43ea1d2931f2 None None] MFA Rules not processed for user `9a67a5c4b7034d22b78331b907cc20bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:22.782964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a2943381-0d25-44f9-bdb0-43ea1d2931f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:22.893548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1229/2462] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:24.197876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905692-e6e4-437c-88ec-9ae8361db7be None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:24.198504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905692-e6e4-437c-88ec-9ae8361db7be None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:24.198683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905692-e6e4-437c-88ec-9ae8361db7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:24.262499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-8a905692-e6e4-437c-88ec-9ae8361db7be None None] MFA Rules not processed for user `57bdca8c562a42a3af75f4d7d36bdecc`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:24.264559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8a905692-e6e4-437c-88ec-9ae8361db7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:24.443276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1234/2463] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:24.455067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2943381-0d25-44f9-bdb0-43ea1d2931f2 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:24.455777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a2943381-0d25-44f9-bdb0-43ea1d2931f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:24.588911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2943381-0d25-44f9-bdb0-43ea1d2931f2 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:24.594671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:24.596340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:24.596723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:24.597097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:24.608357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:24.610023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:24.623341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1230/2464] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:24 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:26.135246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcd705-ae2e-4699-9a42-20fb02abae54 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:26.135733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcd705-ae2e-4699-9a42-20fb02abae54 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:26.135850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcd705-ae2e-4699-9a42-20fb02abae54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:26.154892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-42fcd705-ae2e-4699-9a42-20fb02abae54 None None] MFA Rules not processed for user `57bdca8c562a42a3af75f4d7d36bdecc`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:26.156229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-42fcd705-ae2e-4699-9a42-20fb02abae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:26.168702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1235/2465] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:03:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:26.211991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:26.213565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:26.236555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:26.286932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868f255-0462-467b-8ed2-3a9646441040 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:26.290781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:26.291568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:26.291848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:26.292132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:26.295208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:26.374051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:26.375590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:26.387854 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1231/2466] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:28.310446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcd705-ae2e-4699-9a42-20fb02abae54 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:28.321706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcd705-ae2e-4699-9a42-20fb02abae54 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:28.326688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:28.327350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:28.327636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:28.327909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:28.329999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:28.345592 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:28.347095 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:28.358836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1236/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:34.356969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:34.374889 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910fe95-29e0-4e67-b88c-b63c3e092adc service glance] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:34.392175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:34.393105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:34.393457 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:34.394058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:34.397138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:34.420613 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:34.422364 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:34.434499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1232/2468] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:34.509616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:34.511286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:34.519544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:03:34.523969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d3ddf-1e50-47bb-9a56-a4a5a3c96d77 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:34.529268 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e15f2-9296-479c-b1f0-a429bc91a229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:34.530211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e15f2-9296-479c-b1f0-a429bc91a229 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:34.530567 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e15f2-9296-479c-b1f0-a429bc91a229 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:34.531364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e15f2-9296-479c-b1f0-a429bc91a229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:34.534000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-dd3e15f2-9296-479c-b1f0-a429bc91a229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:34.550456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e15f2-9296-479c-b1f0-a429bc91a229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:34.552188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e15f2-9296-479c-b1f0-a429bc91a229 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:34.562162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1237/2469] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:03:34.646879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:34.661126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5b3e4-7845-423d-aa57-a1d58e7f06e7 service glance] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:34.669184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:34.670267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:34.670547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:34.670813 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:34.672721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:34.688078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:34.689547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:34.701986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1233/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:03:40.512555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c17143-4cbf-41dd-ad60-b312ec8a1dbd None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:40.513199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c17143-4cbf-41dd-ad60-b312ec8a1dbd None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:40.513199 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c17143-4cbf-41dd-ad60-b312ec8a1dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:40.537167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-89c17143-4cbf-41dd-ad60-b312ec8a1dbd None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:03:40.538444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-89c17143-4cbf-41dd-ad60-b312ec8a1dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:40.604547 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1238/2471] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 16:03:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:03:40.620694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:40.622757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:40.703291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479705ae-e342-4dfb-9c6c-4d0053fb7c92 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:40.709226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559e20b-f0bd-4ac8-91ba-6a96ea8cb202 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:40.709954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559e20b-f0bd-4ac8-91ba-6a96ea8cb202 service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:40.710216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559e20b-f0bd-4ac8-91ba-6a96ea8cb202 service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:40.710491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559e20b-f0bd-4ac8-91ba-6a96ea8cb202 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:40.719604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559e20b-f0bd-4ac8-91ba-6a96ea8cb202 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:40.721234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559e20b-f0bd-4ac8-91ba-6a96ea8cb202 service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:40.730126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1234/2472] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 16:03:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 16:03:50.096709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c17143-4cbf-41dd-ad60-b312ec8a1dbd None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:03:50.098671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-89c17143-4cbf-41dd-ad60-b312ec8a1dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:50.116732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c17143-4cbf-41dd-ad60-b312ec8a1dbd None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:03:50.126523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:03:50.127234 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:03:50.127506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:03:50.127797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:03:50.130212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:03:50.239598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:03:50.241198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:03:50.253582 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1239/2473] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:03:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:03:50.726565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=91 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:00.247240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=66 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:01.460888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d045-fc8d-4721-802c-110044a0e953 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:01.461434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d045-fc8d-4721-802c-110044a0e953 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:01.461434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d045-fc8d-4721-802c-110044a0e953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:01.480086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ccb6d045-fc8d-4721-802c-110044a0e953 None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:01.481289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ccb6d045-fc8d-4721-802c-110044a0e953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:01.489884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1235/2474] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jun 21 16:04:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:01.505118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:01.521236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887adb37-37f3-4a42-b72e-57e109cc394c service nova] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:01.529355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e99c0-8853-4a21-a147-f8340ddc0f1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:01.530217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e99c0-8853-4a21-a147-f8340ddc0f1e service swift] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:01.530572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e99c0-8853-4a21-a147-f8340ddc0f1e service swift] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:01.530849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e99c0-8853-4a21-a147-f8340ddc0f1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:01.541457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e99c0-8853-4a21-a147-f8340ddc0f1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:01.543098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e99c0-8853-4a21-a147-f8340ddc0f1e service swift] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:01.553340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1240/2475] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 16:04:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 16:04:02.319996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c805e-7ca7-491d-8fbc-89e471e6b636 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:02.320285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c805e-7ca7-491d-8fbc-89e471e6b636 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:02.320590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c805e-7ca7-491d-8fbc-89e471e6b636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:02.338886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3f7c805e-7ca7-491d-8fbc-89e471e6b636 None None] MFA Rules not processed for user `57bdca8c562a42a3af75f4d7d36bdecc`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:02.340109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3f7c805e-7ca7-491d-8fbc-89e471e6b636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:02.346931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1236/2476] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:05.795076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e99c0-8853-4a21-a147-f8340ddc0f1e service swift] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:05.807248 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e99c0-8853-4a21-a147-f8340ddc0f1e service swift] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:05.811984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:05.813122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:05.813850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:05.814328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:05.817311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:05.829818 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:05.831261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:05.841227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1241/2477] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:12.334405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=6 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:15.836636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=12 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:24.479361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f6481-84d9-4061-87b4-6428db3cf93b None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:24.480015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f6481-84d9-4061-87b4-6428db3cf93b None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:24.480015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f6481-84d9-4061-87b4-6428db3cf93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:24.498899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f31f6481-84d9-4061-87b4-6428db3cf93b None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:24.500150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f31f6481-84d9-4061-87b4-6428db3cf93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:24.509213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1237/2478] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Jun 21 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:24.537849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:24.551988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e9bfc-eedf-42a6-9efe-8c90b56aa76d service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:24.558827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f10bf5-d23e-4ac6-a400-ab071a6844c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:24.559409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f10bf5-d23e-4ac6-a400-ab071a6844c9 service swift] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:24.559693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f10bf5-d23e-4ac6-a400-ab071a6844c9 service swift] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:24.559960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f10bf5-d23e-4ac6-a400-ab071a6844c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:24.568999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f10bf5-d23e-4ac6-a400-ab071a6844c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:24.570678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f10bf5-d23e-4ac6-a400-ab071a6844c9 service swift] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:24.579211 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1242/2479] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 16:04:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 21 16:04:34.493349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:34.576265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=6 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:39.350656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f6481-84d9-4061-87b4-6428db3cf93b None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:39.361907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f6481-84d9-4061-87b4-6428db3cf93b None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:39.369129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:39.369827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:39.370103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:39.370390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:39.372746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:39.388720 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:39.390264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:39.401494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1238/2480] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:40.552391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c13d3-7e72-4160-bd7a-8c22ee49d96a None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:40.552901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c13d3-7e72-4160-bd7a-8c22ee49d96a None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:40.553008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c13d3-7e72-4160-bd7a-8c22ee49d96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:40.572697 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-882c13d3-7e72-4160-bd7a-8c22ee49d96a None None] MFA Rules not processed for user `fd307c87c08b42db9484dbb1c0d223e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:40.573949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-882c13d3-7e72-4160-bd7a-8c22ee49d96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:40.583815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1243/2481] 10.222.0.52 () {58 vars in 967 bytes} [Tue Jun 21 16:04:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:40.596797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:40.608650 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8c374-5408-4b25-aecf-1ed64ee72991 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:40.614197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a519b-aa38-4ba9-8d7e-49b500d0beac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7230dc79eb9844f1aa618d5e1c883b8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:40.614871 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a519b-aa38-4ba9-8d7e-49b500d0beac service swift] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:40.615156 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a519b-aa38-4ba9-8d7e-49b500d0beac service swift] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:40.615621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a519b-aa38-4ba9-8d7e-49b500d0beac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:40.624626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a519b-aa38-4ba9-8d7e-49b500d0beac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:40.626309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a519b-aa38-4ba9-8d7e-49b500d0beac service swift] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:40.634994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1239/2482] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Jun 21 16:04:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 21 16:04:42.756247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c13d3-7e72-4160-bd7a-8c22ee49d96a None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:42.773557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c13d3-7e72-4160-bd7a-8c22ee49d96a None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:42.779363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:42.781679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:42.782195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:42.782542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] PATH_INFO: `/v3/users/57bdca8c562a42a3af75f4d7d36bdecc` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:42.783835 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57bdca8c562a42a3af75f4d7d36bdecc)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:42.785859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:42.857519 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57bdca8c562a42a3af75f4d7d36bdecc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:42.876280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57bdca8c562a42a3af75f4d7d36bdecc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:42.886164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57bdca8c562a42a3af75f4d7d36bdecc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:42.887144 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57bdca8c562a42a3af75f4d7d36bdecc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:42.901508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57bdca8c562a42a3af75f4d7d36bdecc'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:42.911392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1244/2483] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:04:42 2022] DELETE /identity/v3/users/57bdca8c562a42a3af75f4d7d36bdecc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:43.122407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a519b-aa38-4ba9-8d7e-49b500d0beac service swift] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:43.141279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a519b-aa38-4ba9-8d7e-49b500d0beac service swift] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:43.232075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:43.232998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:43.233361 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:43.233648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] PATH_INFO: `/v3/projects/69b62d21e3d54a5592e26594c6483188` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:43.234925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b62d21e3d54a5592e26594c6483188)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:43.237028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:43.299268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b62d21e3d54a5592e26594c6483188', 'request_id': 'req-351451a9-695b-4305-a1ee-e0867eb7265e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.311900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b62d21e3d54a5592e26594c6483188', 'request_id': 'req-351451a9-695b-4305-a1ee-e0867eb7265e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.322778 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b62d21e3d54a5592e26594c6483188', 'request_id': 'req-351451a9-695b-4305-a1ee-e0867eb7265e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.343770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] The token cache is being invalidate because project 69b62d21e3d54a5592e26594c6483188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:04:43.344085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.344699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.346567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1240/2484] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:04:43 2022] DELETE /identity/v3/projects/69b62d21e3d54a5592e26594c6483188 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:43.369067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:43.370588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:43.388770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741ec38-0f66-41c4-a42d-050ba4818a44 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:43.455749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:43.456996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:43.457158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:43.457356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] PATH_INFO: `/v3/users/fa5a48626c934dc2aa4f24002970b6d5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:43.476362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5a48626c934dc2aa4f24002970b6d5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:43.478468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:43.532666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5a48626c934dc2aa4f24002970b6d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.540531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:43.548089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5a48626c934dc2aa4f24002970b6d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.563471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5a48626c934dc2aa4f24002970b6d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.564226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5a48626c934dc2aa4f24002970b6d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.573015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d7f369c-aabb-4439-a088-cfe4285ca07d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5a48626c934dc2aa4f24002970b6d5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.580698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1245/2485] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:04:43 2022] DELETE /identity/v3/users/fa5a48626c934dc2aa4f24002970b6d5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:43.783744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:43.785930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:43.802238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351451a9-695b-4305-a1ee-e0867eb7265e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:43.862015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:43.862894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:43.863172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:43.863444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] PATH_INFO: `/v3/projects/74f35c94371b44c6936b1e713a398f1c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:43.870418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:43.875924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f35c94371b44c6936b1e713a398f1c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:43.878231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:43.932958 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f35c94371b44c6936b1e713a398f1c', 'request_id': 'req-b8ae07a7-619c-4ddc-b22a-0f6339e74408'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.939908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f35c94371b44c6936b1e713a398f1c', 'request_id': 'req-b8ae07a7-619c-4ddc-b22a-0f6339e74408'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.947050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f35c94371b44c6936b1e713a398f1c', 'request_id': 'req-b8ae07a7-619c-4ddc-b22a-0f6339e74408'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.958051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] The token cache is being invalidate because project 74f35c94371b44c6936b1e713a398f1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:04:43.958336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.958963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b8ae07a7-619c-4ddc-b22a-0f6339e74408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:43.960518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1241/2486] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:04:43 2022] DELETE /identity/v3/projects/74f35c94371b44c6936b1e713a398f1c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:43.967753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c3fd-4c75-48ec-9d4a-c0f5d8889fc9 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:43.968229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c3fd-4c75-48ec-9d4a-c0f5d8889fc9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:43.968322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2c3fd-4c75-48ec-9d4a-c0f5d8889fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:43.987459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-9bd2c3fd-4c75-48ec-9d4a-c0f5d8889fc9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:43.988665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9bd2c3fd-4c75-48ec-9d4a-c0f5d8889fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:44.049723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1246/2487] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:44.057828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3e70c-14fa-4730-a51c-baa2380a7bb9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.058123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3e70c-14fa-4730-a51c-baa2380a7bb9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.058397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3e70c-14fa-4730-a51c-baa2380a7bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.077133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8fc3e70c-14fa-4730-a51c-baa2380a7bb9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:44.078617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8fc3e70c-14fa-4730-a51c-baa2380a7bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:44.086701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1242/2488] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:44.092093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2c3fd-4c75-48ec-9d4a-c0f5d8889fc9 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:44.102019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2c3fd-4c75-48ec-9d4a-c0f5d8889fc9 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:44.107072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:44.107771 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.108046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.108339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.109059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:04:44.109364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:44.111041 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:44.123787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1247/2489] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 16:04:44.128237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3e70c-14fa-4730-a51c-baa2380a7bb9 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:44.138245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3e70c-14fa-4730-a51c-baa2380a7bb9 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:44.144126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297b574-00c7-4096-8667-1f4989833600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:44.144798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297b574-00c7-4096-8667-1f4989833600 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.145081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297b574-00c7-4096-8667-1f4989833600 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.145360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0297b574-00c7-4096-8667-1f4989833600 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.146111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297b574-00c7-4096-8667-1f4989833600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:44.148106 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0297b574-00c7-4096-8667-1f4989833600 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:44.178581 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1243/2490] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:04:44 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:44.183297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:44.194387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ebde1-8a58-4a57-8a71-b0bbabf92eff admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:44.261889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28e5b8-e839-4111-81d3-40617ddd8686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:44.262736 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e5b8-e839-4111-81d3-40617ddd8686 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.263023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e5b8-e839-4111-81d3-40617ddd8686 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.263304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e5b8-e839-4111-81d3-40617ddd8686 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.264011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28e5b8-e839-4111-81d3-40617ddd8686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:44.266001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28e5b8-e839-4111-81d3-40617ddd8686 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:44.287066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1248/2491] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:04:44 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:44.291329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297b574-00c7-4096-8667-1f4989833600 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:44.302689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0297b574-00c7-4096-8667-1f4989833600 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:44.307821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b4e04-ede2-4210-b10e-beb4656f5caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:44.308489 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b4e04-ede2-4210-b10e-beb4656f5caf admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.308781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b4e04-ede2-4210-b10e-beb4656f5caf admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.309083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b4e04-ede2-4210-b10e-beb4656f5caf admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.310430 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b4e04-ede2-4210-b10e-beb4656f5caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:44.312123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b4e04-ede2-4210-b10e-beb4656f5caf admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:44.322578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1244/2492] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:44.326982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28e5b8-e839-4111-81d3-40617ddd8686 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:44.337474 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28e5b8-e839-4111-81d3-40617ddd8686 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:44.343685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344a1ca-72f0-4b7b-8266-cda5787bec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:44.344384 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344a1ca-72f0-4b7b-8266-cda5787bec1d admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.344678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344a1ca-72f0-4b7b-8266-cda5787bec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.344987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344a1ca-72f0-4b7b-8266-cda5787bec1d admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.345727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344a1ca-72f0-4b7b-8266-cda5787bec1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:44.347410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344a1ca-72f0-4b7b-8266-cda5787bec1d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:44.358056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1249/2493] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:44.362698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b4e04-ede2-4210-b10e-beb4656f5caf admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:44.373560 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b4e04-ede2-4210-b10e-beb4656f5caf admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:44.379362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:44.380240 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.380546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.380810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] PATH_INFO: `/v3/projects/fe6b9b8c5c0a4a2e92dbb57526264f88/users/01b720a07a5e44bdb7232ff9e5f9be49/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.396586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6b9b8c5c0a4a2e92dbb57526264f88, user_id=01b720a07a5e44bdb7232ff9e5f9be49, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:44.399304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:44.415176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1245/2494] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:04:44 2022] PUT /identity/v3/projects/fe6b9b8c5c0a4a2e92dbb57526264f88/users/01b720a07a5e44bdb7232ff9e5f9be49/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:44.427159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5984da-59d2-4e82-9d34-beddaa8d824c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.427452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5984da-59d2-4e82-9d34-beddaa8d824c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.427735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5984da-59d2-4e82-9d34-beddaa8d824c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.455867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-df5984da-59d2-4e82-9d34-beddaa8d824c None None] MFA Rules not processed for user `01b720a07a5e44bdb7232ff9e5f9be49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:44.457103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-df5984da-59d2-4e82-9d34-beddaa8d824c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:44.489390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:44.527714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1250/2495] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:44.540563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:44.543013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:44.623888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568403d4-8b92-46d7-9f9a-d8e90f3f7aea admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:44.632065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5629e8-a190-47fd-8034-9e6316c17026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:44.632710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5629e8-a190-47fd-8034-9e6316c17026 service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.632993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5629e8-a190-47fd-8034-9e6316c17026 service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.633255 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5629e8-a190-47fd-8034-9e6316c17026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.642336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5629e8-a190-47fd-8034-9e6316c17026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:44.643965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5629e8-a190-47fd-8034-9e6316c17026 service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:44.652984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1246/2496] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:44.870539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5984da-59d2-4e82-9d34-beddaa8d824c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:44.942181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5984da-59d2-4e82-9d34-beddaa8d824c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:44.947467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:44.948576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:44.948576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:44.950267 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] PATH_INFO: `/v3/users/01b720a07a5e44bdb7232ff9e5f9be49` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:44.950819 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b720a07a5e44bdb7232ff9e5f9be49)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:44.952367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:45.018151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b720a07a5e44bdb7232ff9e5f9be49'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.030916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b720a07a5e44bdb7232ff9e5f9be49'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.040284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b720a07a5e44bdb7232ff9e5f9be49'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.040925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b720a07a5e44bdb7232ff9e5f9be49'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.049158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b720a07a5e44bdb7232ff9e5f9be49'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.056324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1251/2497] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:04:44 2022] DELETE /identity/v3/users/01b720a07a5e44bdb7232ff9e5f9be49 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:45.061221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21604a78-e499-4cbc-aa03-cd42b4cdfad9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:45.061527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21604a78-e499-4cbc-aa03-cd42b4cdfad9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:45.061816 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21604a78-e499-4cbc-aa03-cd42b4cdfad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:45.079449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:45.088390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-21604a78-e499-4cbc-aa03-cd42b4cdfad9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:45.091419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-21604a78-e499-4cbc-aa03-cd42b4cdfad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:45.154254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1247/2498] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:45.164127 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:45.165739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:45.222532 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:45.226944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf3ff1-2763-4ab8-b816-a8d5acfefc93 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:45.231967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107dff8-6345-492e-a882-bdf712c19bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:45.233054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107dff8-6345-492e-a882-bdf712c19bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:45.233509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107dff8-6345-492e-a882-bdf712c19bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:45.233868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107dff8-6345-492e-a882-bdf712c19bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:45.244660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107dff8-6345-492e-a882-bdf712c19bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:45.246290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107dff8-6345-492e-a882-bdf712c19bc1 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:45.257351 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1252/2499] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:45.547924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21604a78-e499-4cbc-aa03-cd42b4cdfad9 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:45.549548 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-21604a78-e499-4cbc-aa03-cd42b4cdfad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:45.569696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21604a78-e499-4cbc-aa03-cd42b4cdfad9 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:45.574781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:45.575675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:45.575963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:45.576242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] PATH_INFO: `/v3/projects/fe6b9b8c5c0a4a2e92dbb57526264f88` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:45.577541 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6b9b8c5c0a4a2e92dbb57526264f88)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:45.579607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:45.628741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6b9b8c5c0a4a2e92dbb57526264f88', 'request_id': 'req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.635650 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6b9b8c5c0a4a2e92dbb57526264f88', 'request_id': 'req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.646138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6b9b8c5c0a4a2e92dbb57526264f88', 'request_id': 'req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.658003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] The token cache is being invalidate because project fe6b9b8c5c0a4a2e92dbb57526264f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:04:45.658302 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.658965 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f3ee5663-03dd-46d8-98d9-ba9d628dd42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:45.660573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1248/2500] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:04:45 2022] DELETE /identity/v3/projects/fe6b9b8c5c0a4a2e92dbb57526264f88 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:04:45.665488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d817281-8a4c-4ef8-9506-649f55ac15c1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:45.665792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d817281-8a4c-4ef8-9506-649f55ac15c1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:45.666087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d817281-8a4c-4ef8-9506-649f55ac15c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:45.686213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-7d817281-8a4c-4ef8-9506-649f55ac15c1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:45.688037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-7d817281-8a4c-4ef8-9506-649f55ac15c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:45.754453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1253/2501] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:45.762339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8667d6-5b4b-40b8-99a1-74f83de7e8df None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:45.763185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8667d6-5b4b-40b8-99a1-74f83de7e8df None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:45.763471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8667d6-5b4b-40b8-99a1-74f83de7e8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:45.782970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-4f8667d6-5b4b-40b8-99a1-74f83de7e8df None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:45.784187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-4f8667d6-5b4b-40b8-99a1-74f83de7e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:45.792380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1249/2502] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:45.797100 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d817281-8a4c-4ef8-9506-649f55ac15c1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:45.807429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d817281-8a4c-4ef8-9506-649f55ac15c1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:45.812706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:45.813363 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:45.813648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:45.813963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:45.814691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:04:45.814992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:45.816648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:45.829238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1254/2503] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:04:45.834062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8667d6-5b4b-40b8-99a1-74f83de7e8df None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:45.845383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8667d6-5b4b-40b8-99a1-74f83de7e8df None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:45.851283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6117463-b8ec-4971-b4dd-7bf0597059b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:45.851944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117463-b8ec-4971-b4dd-7bf0597059b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:45.852226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117463-b8ec-4971-b4dd-7bf0597059b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:45.852497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6117463-b8ec-4971-b4dd-7bf0597059b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:45.853193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6117463-b8ec-4971-b4dd-7bf0597059b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:45.855148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6117463-b8ec-4971-b4dd-7bf0597059b4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:45.886070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1250/2504] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:04:45 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:45.887246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:45.901856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09c32e-0707-4a69-8b06-949fc82579d8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:45.981537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba76ece-3910-427c-bb22-ec5c3dd2f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:45.982278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba76ece-3910-427c-bb22-ec5c3dd2f62a admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:45.982536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba76ece-3910-427c-bb22-ec5c3dd2f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:45.982904 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba76ece-3910-427c-bb22-ec5c3dd2f62a admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:45.985010 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba76ece-3910-427c-bb22-ec5c3dd2f62a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:45.985458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba76ece-3910-427c-bb22-ec5c3dd2f62a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.008232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1255/2505] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:04:45 2022] POST /identity/v3/users => generated 284 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:46.012113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6117463-b8ec-4971-b4dd-7bf0597059b4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.030826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6117463-b8ec-4971-b4dd-7bf0597059b4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.039797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b299642-3d3f-44bf-ad21-b459cf05920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.040481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299642-3d3f-44bf-ad21-b459cf05920f admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.040757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299642-3d3f-44bf-ad21-b459cf05920f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.041032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b299642-3d3f-44bf-ad21-b459cf05920f admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.041767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b299642-3d3f-44bf-ad21-b459cf05920f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.043481 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b299642-3d3f-44bf-ad21-b459cf05920f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.055145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1251/2506] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:46.059630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba76ece-3910-427c-bb22-ec5c3dd2f62a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.069816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba76ece-3910-427c-bb22-ec5c3dd2f62a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.075053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20d479-27f0-4c94-afc7-3005a7b98624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.075691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20d479-27f0-4c94-afc7-3005a7b98624 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.076225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20d479-27f0-4c94-afc7-3005a7b98624 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.076639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20d479-27f0-4c94-afc7-3005a7b98624 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.077537 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20d479-27f0-4c94-afc7-3005a7b98624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.079666 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20d479-27f0-4c94-afc7-3005a7b98624 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.090915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1256/2507] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:46.095671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b299642-3d3f-44bf-ad21-b459cf05920f admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.106008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b299642-3d3f-44bf-ad21-b459cf05920f admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.111167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcbaed-2636-486a-9380-2ca9df8be19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.112052 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcbaed-2636-486a-9380-2ca9df8be19c admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.112326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcbaed-2636-486a-9380-2ca9df8be19c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.112601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcbaed-2636-486a-9380-2ca9df8be19c admin admin] PATH_INFO: `/v3/projects/dbf4aabda883489ea168546bd799492b/users/5ee86851540345d9ad5358019e332068/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.129396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcbaed-2636-486a-9380-2ca9df8be19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf4aabda883489ea168546bd799492b, user_id=5ee86851540345d9ad5358019e332068, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.132088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcbaed-2636-486a-9380-2ca9df8be19c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.147177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1252/2508] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:04:46 2022] PUT /identity/v3/projects/dbf4aabda883489ea168546bd799492b/users/5ee86851540345d9ad5358019e332068/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:46.158512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb7f4-c2e0-4e1b-8109-323b9df75d15 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.158803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb7f4-c2e0-4e1b-8109-323b9df75d15 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.159098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbb7f4-c2e0-4e1b-8109-323b9df75d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.188121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d4bbb7f4-c2e0-4e1b-8109-323b9df75d15 None None] MFA Rules not processed for user `5ee86851540345d9ad5358019e332068`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:46.189339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d4bbb7f4-c2e0-4e1b-8109-323b9df75d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:46.237076 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:46.268475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1257/2509] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:46.273272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcbaed-2636-486a-9380-2ca9df8be19c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.307575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:46.344762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcbaed-2636-486a-9380-2ca9df8be19c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.350419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff57dec-d608-4ef3-908b-c2eba74ad34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.351113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff57dec-d608-4ef3-908b-c2eba74ad34c admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.351398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff57dec-d608-4ef3-908b-c2eba74ad34c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.351671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff57dec-d608-4ef3-908b-c2eba74ad34c admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.352387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff57dec-d608-4ef3-908b-c2eba74ad34c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.354802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff57dec-d608-4ef3-908b-c2eba74ad34c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.389217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1253/2510] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:04:46 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:46.393650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbb7f4-c2e0-4e1b-8109-323b9df75d15 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.405596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbb7f4-c2e0-4e1b-8109-323b9df75d15 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.472835 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721c848-19e6-41bc-a769-4257500cf55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.473523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721c848-19e6-41bc-a769-4257500cf55b admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.473830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721c848-19e6-41bc-a769-4257500cf55b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.474152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721c848-19e6-41bc-a769-4257500cf55b admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.474825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721c848-19e6-41bc-a769-4257500cf55b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.476788 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721c848-19e6-41bc-a769-4257500cf55b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.505208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1258/2511] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:04:46 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:46.509884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff57dec-d608-4ef3-908b-c2eba74ad34c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.521696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff57dec-d608-4ef3-908b-c2eba74ad34c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.527382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c3966-306c-4098-9483-324ce3b07566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.528090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c3966-306c-4098-9483-324ce3b07566 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.528525 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c3966-306c-4098-9483-324ce3b07566 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.529023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c3966-306c-4098-9483-324ce3b07566 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.529492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c3966-306c-4098-9483-324ce3b07566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.531188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c3966-306c-4098-9483-324ce3b07566 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.543031 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1254/2512] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:46.547969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721c848-19e6-41bc-a769-4257500cf55b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.558973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721c848-19e6-41bc-a769-4257500cf55b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.564249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cae51-6bbb-40e8-8236-1edebcdc234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.565167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cae51-6bbb-40e8-8236-1edebcdc234e admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.565467 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cae51-6bbb-40e8-8236-1edebcdc234e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.565760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cae51-6bbb-40e8-8236-1edebcdc234e admin admin] PATH_INFO: `/v3/projects/3e86eb4c6e4f4b55b82b8907b889092c/users/952c7a1e155241cdbc9997456eca9eca/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.587903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cae51-6bbb-40e8-8236-1edebcdc234e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e86eb4c6e4f4b55b82b8907b889092c, user_id=952c7a1e155241cdbc9997456eca9eca, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.591037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cae51-6bbb-40e8-8236-1edebcdc234e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.605683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1259/2513] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:04:46 2022] PUT /identity/v3/projects/3e86eb4c6e4f4b55b82b8907b889092c/users/952c7a1e155241cdbc9997456eca9eca/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:46.610045 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c3966-306c-4098-9483-324ce3b07566 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.621320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c3966-306c-4098-9483-324ce3b07566 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.689983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55353de7-7c68-4348-87a7-2bdc0761390a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.690932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55353de7-7c68-4348-87a7-2bdc0761390a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.691332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55353de7-7c68-4348-87a7-2bdc0761390a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.691686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55353de7-7c68-4348-87a7-2bdc0761390a admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.692777 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55353de7-7c68-4348-87a7-2bdc0761390a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.694980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55353de7-7c68-4348-87a7-2bdc0761390a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.707904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1255/2514] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:46.712381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cae51-6bbb-40e8-8236-1edebcdc234e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.728880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cae51-6bbb-40e8-8236-1edebcdc234e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.735007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.735928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.736204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.736486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] PATH_INFO: `/v3/projects/3e86eb4c6e4f4b55b82b8907b889092c/users/952c7a1e155241cdbc9997456eca9eca/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.738615 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e86eb4c6e4f4b55b82b8907b889092c, user_id=952c7a1e155241cdbc9997456eca9eca, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.741379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.752613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1260/2515] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:04:46 2022] PUT /identity/v3/projects/3e86eb4c6e4f4b55b82b8907b889092c/users/952c7a1e155241cdbc9997456eca9eca/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:46.764024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3f829-2140-4fb4-a236-9268a34a3462 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.764316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3f829-2140-4fb4-a236-9268a34a3462 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.764589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3f829-2140-4fb4-a236-9268a34a3462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.794464 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-02b3f829-2140-4fb4-a236-9268a34a3462 None None] MFA Rules not processed for user `952c7a1e155241cdbc9997456eca9eca`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:46.795780 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-02b3f829-2140-4fb4-a236-9268a34a3462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:46.873355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1256/2516] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:46.888049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:46.890740 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:46.959341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81d095-d90c-4ecf-802b-fbb826358a9c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:46.963540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd134ad-4126-40ab-88c3-821379e77ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:46.964514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd134ad-4126-40ab-88c3-821379e77ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:46.964796 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd134ad-4126-40ab-88c3-821379e77ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:46.965074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd134ad-4126-40ab-88c3-821379e77ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:46.975075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd134ad-4126-40ab-88c3-821379e77ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:46.976533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd134ad-4126-40ab-88c3-821379e77ea3 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:46.987391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1261/2517] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:47.039071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3f829-2140-4fb4-a236-9268a34a3462 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:47.040499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-02b3f829-2140-4fb4-a236-9268a34a3462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:47.053709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3f829-2140-4fb4-a236-9268a34a3462 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:47.058007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45b13a-10ec-4c3a-ba43-81750d857695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:47.058656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45b13a-10ec-4c3a-ba43-81750d857695 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:47.058936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45b13a-10ec-4c3a-ba43-81750d857695 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:47.059203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45b13a-10ec-4c3a-ba43-81750d857695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:47.082766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:47.134851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45b13a-10ec-4c3a-ba43-81750d857695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:47.136316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45b13a-10ec-4c3a-ba43-81750d857695 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:47.147224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1257/2518] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:04:50.596364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd134ad-4126-40ab-88c3-821379e77ea3 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:50.651566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:50.677063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd134ad-4126-40ab-88c3-821379e77ea3 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:50.683509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:50.684596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:50.684990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:50.685282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] PATH_INFO: `/v3/users/5ee86851540345d9ad5358019e332068` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:50.687208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee86851540345d9ad5358019e332068)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:50.689443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:50.755107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee86851540345d9ad5358019e332068'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:50.768583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee86851540345d9ad5358019e332068'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:50.777816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee86851540345d9ad5358019e332068'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:50.778981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee86851540345d9ad5358019e332068'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:50.787077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee86851540345d9ad5358019e332068'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:50.793389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1262/2519] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:04:50 2022] DELETE /identity/v3/users/5ee86851540345d9ad5358019e332068 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:50.798267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaec90-2070-465e-9a47-b322475b1d87 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:50.798756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaec90-2070-465e-9a47-b322475b1d87 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:50.798849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaec90-2070-465e-9a47-b322475b1d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:50.816551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-73eaec90-2070-465e-9a47-b322475b1d87 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:50.817808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-73eaec90-2070-465e-9a47-b322475b1d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:50.889980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1258/2520] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:50.899977 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:50.901455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:50.966841 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82b0d7-d53d-4548-bf42-002199c95ef4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:50.971074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9256881-f1c2-4566-b75a-0a26f672d6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:50.971745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9256881-f1c2-4566-b75a-0a26f672d6bf service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:50.972030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9256881-f1c2-4566-b75a-0a26f672d6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:50.972308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9256881-f1c2-4566-b75a-0a26f672d6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:50.982639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9256881-f1c2-4566-b75a-0a26f672d6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:50.984228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9256881-f1c2-4566-b75a-0a26f672d6bf service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:50.994933 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1263/2521] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:51.297235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaec90-2070-465e-9a47-b322475b1d87 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:51.299219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-73eaec90-2070-465e-9a47-b322475b1d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:51.312675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaec90-2070-465e-9a47-b322475b1d87 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:51.318883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:51.319808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:51.320093 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:51.320443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] PATH_INFO: `/v3/projects/dbf4aabda883489ea168546bd799492b` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:51.321860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf4aabda883489ea168546bd799492b)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:51.323750 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:51.379847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf4aabda883489ea168546bd799492b', 'request_id': 'req-0a34ba00-fab5-48e9-aad5-9710990ae5bb'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.386663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf4aabda883489ea168546bd799492b', 'request_id': 'req-0a34ba00-fab5-48e9-aad5-9710990ae5bb'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.397099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf4aabda883489ea168546bd799492b', 'request_id': 'req-0a34ba00-fab5-48e9-aad5-9710990ae5bb'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.408320 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] The token cache is being invalidate because project dbf4aabda883489ea168546bd799492b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:04:51.408559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.409276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.410954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1259/2522] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:04:51 2022] DELETE /identity/v3/projects/dbf4aabda883489ea168546bd799492b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:51.415035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9256881-f1c2-4566-b75a-0a26f672d6bf service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:51.425754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9256881-f1c2-4566-b75a-0a26f672d6bf service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:51.505664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:51.507678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:51.507986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:51.508356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] PATH_INFO: `/v3/users/952c7a1e155241cdbc9997456eca9eca` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:51.510367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=952c7a1e155241cdbc9997456eca9eca)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:51.513408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:51.518779 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:51.570902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952c7a1e155241cdbc9997456eca9eca'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.583187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952c7a1e155241cdbc9997456eca9eca'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.592039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952c7a1e155241cdbc9997456eca9eca'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.592698 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952c7a1e155241cdbc9997456eca9eca'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.600543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d161af5b-7bb2-49d3-be11-06dd68db0c7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952c7a1e155241cdbc9997456eca9eca'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:51.606830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1264/2523] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:04:51 2022] DELETE /identity/v3/users/952c7a1e155241cdbc9997456eca9eca => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:51.911070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:51.912615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:51.927636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34ba00-fab5-48e9-aad5-9710990ae5bb admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:51.973177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:51.990424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:51.991193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:51.991474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:51.991749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] PATH_INFO: `/v3/projects/3e86eb4c6e4f4b55b82b8907b889092c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:51.992973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e86eb4c6e4f4b55b82b8907b889092c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:51.994946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.044118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e86eb4c6e4f4b55b82b8907b889092c', 'request_id': 'req-711c2892-4d5e-424c-8a2e-46d6555b1c16'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:52.050594 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e86eb4c6e4f4b55b82b8907b889092c', 'request_id': 'req-711c2892-4d5e-424c-8a2e-46d6555b1c16'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:52.059006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e86eb4c6e4f4b55b82b8907b889092c', 'request_id': 'req-711c2892-4d5e-424c-8a2e-46d6555b1c16'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:52.070827 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] The token cache is being invalidate because project 3e86eb4c6e4f4b55b82b8907b889092c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:04:52.071114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:52.071775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-711c2892-4d5e-424c-8a2e-46d6555b1c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:52.073394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1260/2524] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:04:51 2022] DELETE /identity/v3/projects/3e86eb4c6e4f4b55b82b8907b889092c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:52.079586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526e87-014a-4231-8beb-fc64c7929563 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.079840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526e87-014a-4231-8beb-fc64c7929563 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.080078 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526e87-014a-4231-8beb-fc64c7929563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.102766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-1d526e87-014a-4231-8beb-fc64c7929563 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:52.103987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1d526e87-014a-4231-8beb-fc64c7929563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:52.174874 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1265/2525] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:04:52.183851 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd555b-91a5-41f5-a7d3-d9c7586b95d3 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.184262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd555b-91a5-41f5-a7d3-d9c7586b95d3 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.184644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd555b-91a5-41f5-a7d3-d9c7586b95d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.216986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-33fd555b-91a5-41f5-a7d3-d9c7586b95d3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:52.218235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-33fd555b-91a5-41f5-a7d3-d9c7586b95d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:52.226483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1261/2526] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:52.230875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d526e87-014a-4231-8beb-fc64c7929563 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.242293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d526e87-014a-4231-8beb-fc64c7929563 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.247478 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.248378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.248588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.248875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.249624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:04:52.249982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.251609 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.264930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1266/2527] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:04:52.269599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd555b-91a5-41f5-a7d3-d9c7586b95d3 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.279260 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd555b-91a5-41f5-a7d3-d9c7586b95d3 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.284494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcc8ce-d6f6-4cb4-b3b3-dd215688cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.285154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcc8ce-d6f6-4cb4-b3b3-dd215688cd94 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.285438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcc8ce-d6f6-4cb4-b3b3-dd215688cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.285695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcc8ce-d6f6-4cb4-b3b3-dd215688cd94 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.286432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcc8ce-d6f6-4cb4-b3b3-dd215688cd94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.288592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcc8ce-d6f6-4cb4-b3b3-dd215688cd94 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.320511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1262/2528] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:04:52 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:52.326574 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.338880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dcd3f-381a-4c69-9bf8-11751e6c9e0f admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.407542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f36f7d-2a5a-48c8-98e2-44beb95efea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.408447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36f7d-2a5a-48c8-98e2-44beb95efea7 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.408644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36f7d-2a5a-48c8-98e2-44beb95efea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.408921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36f7d-2a5a-48c8-98e2-44beb95efea7 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.409632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f36f7d-2a5a-48c8-98e2-44beb95efea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.411676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f36f7d-2a5a-48c8-98e2-44beb95efea7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.429394 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:52.437458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1267/2529] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:04:52 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:52.441768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcc8ce-d6f6-4cb4-b3b3-dd215688cd94 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.452667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcc8ce-d6f6-4cb4-b3b3-dd215688cd94 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.459894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0180d-d1a1-434f-9bc8-3d1a721b25e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.460622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0180d-d1a1-434f-9bc8-3d1a721b25e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.461015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0180d-d1a1-434f-9bc8-3d1a721b25e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.461329 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0180d-d1a1-434f-9bc8-3d1a721b25e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.462161 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0180d-d1a1-434f-9bc8-3d1a721b25e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.463904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0180d-d1a1-434f-9bc8-3d1a721b25e2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.480909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1263/2530] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:04:52.485434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f36f7d-2a5a-48c8-98e2-44beb95efea7 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.499337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f36f7d-2a5a-48c8-98e2-44beb95efea7 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.504358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd744ed-fd79-43de-9d02-dad48d982339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.505000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd744ed-fd79-43de-9d02-dad48d982339 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.505281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd744ed-fd79-43de-9d02-dad48d982339 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.505555 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd744ed-fd79-43de-9d02-dad48d982339 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.506326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd744ed-fd79-43de-9d02-dad48d982339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.507948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd744ed-fd79-43de-9d02-dad48d982339 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.520296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1268/2531] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:52.526000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0180d-d1a1-434f-9bc8-3d1a721b25e2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.539761 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0180d-d1a1-434f-9bc8-3d1a721b25e2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.548765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b3532-bafc-4938-90ee-6de30b521157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.549987 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b3532-bafc-4938-90ee-6de30b521157 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.550293 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b3532-bafc-4938-90ee-6de30b521157 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.550698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b3532-bafc-4938-90ee-6de30b521157 admin admin] PATH_INFO: `/v3/projects/ddcb21b028054d6082d5ce66630f8648/users/e115432b568749e7be4a7b544ba350ad/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.567977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b3532-bafc-4938-90ee-6de30b521157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddcb21b028054d6082d5ce66630f8648, user_id=e115432b568749e7be4a7b544ba350ad, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.570766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b3532-bafc-4938-90ee-6de30b521157 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.586573 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1264/2532] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:04:52 2022] PUT /identity/v3/projects/ddcb21b028054d6082d5ce66630f8648/users/e115432b568749e7be4a7b544ba350ad/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:52.600959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c04f1-eac8-44a5-b26a-3984ce61df04 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.601528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c04f1-eac8-44a5-b26a-3984ce61df04 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.601528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c04f1-eac8-44a5-b26a-3984ce61df04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.631365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b33c04f1-eac8-44a5-b26a-3984ce61df04 None None] MFA Rules not processed for user `e115432b568749e7be4a7b544ba350ad`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:52.632961 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b33c04f1-eac8-44a5-b26a-3984ce61df04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:52.704261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1269/2533] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:52.711115 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b3532-bafc-4938-90ee-6de30b521157 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.779972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b3532-bafc-4938-90ee-6de30b521157 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.785328 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72302992-1ac2-4073-bab1-0fcd7f1cbc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.786035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72302992-1ac2-4073-bab1-0fcd7f1cbc8b admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.786321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72302992-1ac2-4073-bab1-0fcd7f1cbc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.786603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72302992-1ac2-4073-bab1-0fcd7f1cbc8b admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.787309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72302992-1ac2-4073-bab1-0fcd7f1cbc8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.789279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72302992-1ac2-4073-bab1-0fcd7f1cbc8b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.804294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:52.820043 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1265/2534] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:04:52 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:52.824377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c04f1-eac8-44a5-b26a-3984ce61df04 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.836168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c04f1-eac8-44a5-b26a-3984ce61df04 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.901131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0de711-a92b-4b41-a7ad-dc656a613c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.901723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0de711-a92b-4b41-a7ad-dc656a613c1c admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.902024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0de711-a92b-4b41-a7ad-dc656a613c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.902305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0de711-a92b-4b41-a7ad-dc656a613c1c admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.902996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0de711-a92b-4b41-a7ad-dc656a613c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.905524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0de711-a92b-4b41-a7ad-dc656a613c1c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.931339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1270/2535] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:04:52 2022] POST /identity/v3/users => generated 307 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:52.935439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72302992-1ac2-4073-bab1-0fcd7f1cbc8b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.946678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72302992-1ac2-4073-bab1-0fcd7f1cbc8b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.952532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b53e9-e4e3-4885-ba28-136c2be354a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.953382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b53e9-e4e3-4885-ba28-136c2be354a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.953663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b53e9-e4e3-4885-ba28-136c2be354a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.953974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b53e9-e4e3-4885-ba28-136c2be354a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:52.954703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b53e9-e4e3-4885-ba28-136c2be354a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:52.956310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b53e9-e4e3-4885-ba28-136c2be354a2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:52.968198 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1266/2536] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:52.972743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0de711-a92b-4b41-a7ad-dc656a613c1c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:52.984931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0de711-a92b-4b41-a7ad-dc656a613c1c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:52.990652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b40024-679e-428c-b5a9-8b10a860bd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:52.991726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40024-679e-428c-b5a9-8b10a860bd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:52.992105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40024-679e-428c-b5a9-8b10a860bd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:52.992466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b40024-679e-428c-b5a9-8b10a860bd73 admin admin] PATH_INFO: `/v3/projects/db16eacc0da44afba43eae71e5d6e654/users/c08af5c6c736482986856ca45ebaaaf8/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:53.010924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b40024-679e-428c-b5a9-8b10a860bd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db16eacc0da44afba43eae71e5d6e654, user_id=c08af5c6c736482986856ca45ebaaaf8, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:53.013762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b40024-679e-428c-b5a9-8b10a860bd73 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:53.029711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1271/2537] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:04:52 2022] PUT /identity/v3/projects/db16eacc0da44afba43eae71e5d6e654/users/c08af5c6c736482986856ca45ebaaaf8/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:53.033831 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b53e9-e4e3-4885-ba28-136c2be354a2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:53.047252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b53e9-e4e3-4885-ba28-136c2be354a2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:53.111977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7d1f9-a15e-4de5-8a49-d548858031fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:53.112692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7d1f9-a15e-4de5-8a49-d548858031fc admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:53.112976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7d1f9-a15e-4de5-8a49-d548858031fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:53.113244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7d1f9-a15e-4de5-8a49-d548858031fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:53.114040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7d1f9-a15e-4de5-8a49-d548858031fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:53.115625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7d1f9-a15e-4de5-8a49-d548858031fc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:53.126631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1267/2538] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:04:53.130914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b40024-679e-428c-b5a9-8b10a860bd73 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:53.141653 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b40024-679e-428c-b5a9-8b10a860bd73 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:53.146386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:53.147456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:53.147661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:53.147936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] PATH_INFO: `/v3/projects/db16eacc0da44afba43eae71e5d6e654/users/c08af5c6c736482986856ca45ebaaaf8/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:53.149996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db16eacc0da44afba43eae71e5d6e654, user_id=c08af5c6c736482986856ca45ebaaaf8, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:53.152725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:53.164746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1272/2539] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:04:53 2022] PUT /identity/v3/projects/db16eacc0da44afba43eae71e5d6e654/users/c08af5c6c736482986856ca45ebaaaf8/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:53.175807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17f50-75e7-45b9-842b-32dd0c669623 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:53.175807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17f50-75e7-45b9-842b-32dd0c669623 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:53.176141 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c17f50-75e7-45b9-842b-32dd0c669623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:53.204692 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b6c17f50-75e7-45b9-842b-32dd0c669623 None None] MFA Rules not processed for user `c08af5c6c736482986856ca45ebaaaf8`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:53.206016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b6c17f50-75e7-45b9-842b-32dd0c669623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:53.292204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1268/2540] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:04:53.304241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:53.306049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:53.350939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:53.379787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cede254-7a4c-495f-a8f3-6ae3ce27a1e2 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:53.383543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3e412-5151-4ebf-a938-4f6cc88cec20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:53.384196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3e412-5151-4ebf-a938-4f6cc88cec20 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:53.384477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3e412-5151-4ebf-a938-4f6cc88cec20 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:53.384752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3e412-5151-4ebf-a938-4f6cc88cec20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:53.394584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3e412-5151-4ebf-a938-4f6cc88cec20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:53.396017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3e412-5151-4ebf-a938-4f6cc88cec20 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:53.405935 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1273/2541] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:53 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:53.485593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17f50-75e7-45b9-842b-32dd0c669623 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:53.496344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c17f50-75e7-45b9-842b-32dd0c669623 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:53.501540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b5ca1-c1f1-4a5c-913c-d632bdc0db21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:53.502482 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5ca1-c1f1-4a5c-913c-d632bdc0db21 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:53.502787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5ca1-c1f1-4a5c-913c-d632bdc0db21 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:53.503196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5ca1-c1f1-4a5c-913c-d632bdc0db21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:53.571924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b5ca1-c1f1-4a5c-913c-d632bdc0db21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:53.573376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b5ca1-c1f1-4a5c-913c-d632bdc0db21 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:53.583129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1269/2542] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:57.718297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3e412-5151-4ebf-a938-4f6cc88cec20 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:57.792036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3e412-5151-4ebf-a938-4f6cc88cec20 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:57.798848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:57.799980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:57.800286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:57.800669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] PATH_INFO: `/v3/users/e115432b568749e7be4a7b544ba350ad` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:57.802578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e115432b568749e7be4a7b544ba350ad)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:57.805172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:57.859332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e115432b568749e7be4a7b544ba350ad'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:57.871957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e115432b568749e7be4a7b544ba350ad'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:57.880649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e115432b568749e7be4a7b544ba350ad'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:57.881366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e115432b568749e7be4a7b544ba350ad'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:57.889151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e115432b568749e7be4a7b544ba350ad'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:57.897022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1274/2543] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:04:57 2022] DELETE /identity/v3/users/e115432b568749e7be4a7b544ba350ad => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:57.902290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2b2c0-8baa-4896-9c77-0667d40fae8e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:57.902836 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2b2c0-8baa-4896-9c77-0667d40fae8e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:57.902836 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2b2c0-8baa-4896-9c77-0667d40fae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:57.911496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:57.922975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-fdb2b2c0-8baa-4896-9c77-0667d40fae8e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:57.924335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fdb2b2c0-8baa-4896-9c77-0667d40fae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:57.992105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1270/2544] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:04:58.003446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:58.005120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:58.070565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc026f-856e-4371-9ad0-824ce24f3131 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:58.074344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38815958-75bd-414e-9863-7d879d68a2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:58.075006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38815958-75bd-414e-9863-7d879d68a2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:58.075289 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38815958-75bd-414e-9863-7d879d68a2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:58.075573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38815958-75bd-414e-9863-7d879d68a2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:58.083630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:58.087165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38815958-75bd-414e-9863-7d879d68a2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:58.090805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38815958-75bd-414e-9863-7d879d68a2a0 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:58.102800 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1275/2545] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:04:58.397330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2b2c0-8baa-4896-9c77-0667d40fae8e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:58.401237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fdb2b2c0-8baa-4896-9c77-0667d40fae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:58.418659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2b2c0-8baa-4896-9c77-0667d40fae8e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:58.425016 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:58.425940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:58.426221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:58.426726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] PATH_INFO: `/v3/projects/ddcb21b028054d6082d5ce66630f8648` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:58.428073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddcb21b028054d6082d5ce66630f8648)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:58.430094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:58.489955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddcb21b028054d6082d5ce66630f8648', 'request_id': 'req-b26fff2d-f510-40ab-8c5e-323db1336d4b'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.496494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddcb21b028054d6082d5ce66630f8648', 'request_id': 'req-b26fff2d-f510-40ab-8c5e-323db1336d4b'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.505236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddcb21b028054d6082d5ce66630f8648', 'request_id': 'req-b26fff2d-f510-40ab-8c5e-323db1336d4b'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.519865 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] The token cache is being invalidate because project ddcb21b028054d6082d5ce66630f8648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:04:58.520162 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.520849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.522515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1271/2546] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:04:58 2022] DELETE /identity/v3/projects/ddcb21b028054d6082d5ce66630f8648 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:58.526868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38815958-75bd-414e-9863-7d879d68a2a0 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:58.537932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38815958-75bd-414e-9863-7d879d68a2a0 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:58.609381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:58.610380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:58.610775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:58.611172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] PATH_INFO: `/v3/users/c08af5c6c736482986856ca45ebaaaf8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:58.612393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08af5c6c736482986856ca45ebaaaf8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:58.614665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:58.675706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08af5c6c736482986856ca45ebaaaf8'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.690339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08af5c6c736482986856ca45ebaaaf8'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.699509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08af5c6c736482986856ca45ebaaaf8'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.700209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08af5c6c736482986856ca45ebaaaf8'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.708556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b5ac0cf5-e246-4fe1-9993-666e00e110ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08af5c6c736482986856ca45ebaaaf8'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:58.714923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1276/2547] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:04:58 2022] DELETE /identity/v3/users/c08af5c6c736482986856ca45ebaaaf8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:59.014721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.017308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:59.033901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fff2d-f510-40ab-8c5e-323db1336d4b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:59.104216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:59.105069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.105317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.105688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] PATH_INFO: `/v3/projects/db16eacc0da44afba43eae71e5d6e654` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.106986 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db16eacc0da44afba43eae71e5d6e654)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:59.109215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:59.155439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:59.179656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db16eacc0da44afba43eae71e5d6e654', 'request_id': 'req-ec01fa43-2453-4900-bf41-1c43759d4469'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:59.187785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db16eacc0da44afba43eae71e5d6e654', 'request_id': 'req-ec01fa43-2453-4900-bf41-1c43759d4469'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:59.196159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db16eacc0da44afba43eae71e5d6e654', 'request_id': 'req-ec01fa43-2453-4900-bf41-1c43759d4469'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:59.207589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] The token cache is being invalidate because project db16eacc0da44afba43eae71e5d6e654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:04:59.207875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:59.208590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ec01fa43-2453-4900-bf41-1c43759d4469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:04:59.210334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1272/2548] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:04:59 2022] DELETE /identity/v3/projects/db16eacc0da44afba43eae71e5d6e654 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:59.218671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01066df8-d50f-4471-aaa9-85c861b81c0e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.219561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01066df8-d50f-4471-aaa9-85c861b81c0e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.219561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01066df8-d50f-4471-aaa9-85c861b81c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.239997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-01066df8-d50f-4471-aaa9-85c861b81c0e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:59.241710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-01066df8-d50f-4471-aaa9-85c861b81c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:59.312867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1277/2549] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:59.321358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8f20-ca6c-4e21-83b6-d3b3d0e4f11f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.321673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8f20-ca6c-4e21-83b6-d3b3d0e4f11f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.321988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c8f20-ca6c-4e21-83b6-d3b3d0e4f11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.343566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2f5c8f20-ca6c-4e21-83b6-d3b3d0e4f11f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:59.344800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2f5c8f20-ca6c-4e21-83b6-d3b3d0e4f11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:59.354630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1273/2550] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:04:59.360139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01066df8-d50f-4471-aaa9-85c861b81c0e None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.370659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01066df8-d50f-4471-aaa9-85c861b81c0e None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:59.376514 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:59.377207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.377492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.377797 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.378541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:04:59.378861 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:59.380475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:59.392620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1278/2551] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:04:59.397200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c8f20-ca6c-4e21-83b6-d3b3d0e4f11f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.406839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c8f20-ca6c-4e21-83b6-d3b3d0e4f11f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:59.411673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d2ee4-ef6f-4186-a8ab-f49990d2ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:59.412330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d2ee4-ef6f-4186-a8ab-f49990d2ca75 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.412612 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d2ee4-ef6f-4186-a8ab-f49990d2ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.412885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d2ee4-ef6f-4186-a8ab-f49990d2ca75 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.413604 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d2ee4-ef6f-4186-a8ab-f49990d2ca75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:59.415559 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d2ee4-ef6f-4186-a8ab-f49990d2ca75 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:59.442611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1274/2552] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:04:59 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:59.446807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.452191 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:04:59.461497 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f69de-f015-4215-812c-601b041d7e11 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:59.525122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f0905-45a4-45c0-9e49-8573202786ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:59.525907 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f0905-45a4-45c0-9e49-8573202786ce admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.526239 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f0905-45a4-45c0-9e49-8573202786ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.526559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f0905-45a4-45c0-9e49-8573202786ce admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.527393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f0905-45a4-45c0-9e49-8573202786ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:59.529734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f0905-45a4-45c0-9e49-8573202786ce admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:59.551301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1279/2553] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:04:59 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:04:59.555434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d2ee4-ef6f-4186-a8ab-f49990d2ca75 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.566799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d2ee4-ef6f-4186-a8ab-f49990d2ca75 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:59.572311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8d94a-5ceb-4769-aeef-a14a163b613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:59.573009 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8d94a-5ceb-4769-aeef-a14a163b613e admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.573294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8d94a-5ceb-4769-aeef-a14a163b613e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.573569 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8d94a-5ceb-4769-aeef-a14a163b613e admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.574334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8d94a-5ceb-4769-aeef-a14a163b613e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:59.575977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8d94a-5ceb-4769-aeef-a14a163b613e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:59.587007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1275/2554] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:04:59.591580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f0905-45a4-45c0-9e49-8573202786ce admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.602048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f0905-45a4-45c0-9e49-8573202786ce admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:59.607536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b32cbc-845f-4584-b3d8-4adaae8d6f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:59.608217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b32cbc-845f-4584-b3d8-4adaae8d6f48 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.608598 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b32cbc-845f-4584-b3d8-4adaae8d6f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.608876 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b32cbc-845f-4584-b3d8-4adaae8d6f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.609601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b32cbc-845f-4584-b3d8-4adaae8d6f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:59.611637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b32cbc-845f-4584-b3d8-4adaae8d6f48 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:59.624632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1280/2555] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:04:59.630810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8d94a-5ceb-4769-aeef-a14a163b613e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.647680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8d94a-5ceb-4769-aeef-a14a163b613e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:59.652533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f21c2-0baf-4469-a43f-97188d00dfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:59.653418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f21c2-0baf-4469-a43f-97188d00dfdc admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.653694 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f21c2-0baf-4469-a43f-97188d00dfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.653997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f21c2-0baf-4469-a43f-97188d00dfdc admin admin] PATH_INFO: `/v3/projects/6655fbbd6fbd4335b2147f0f066bf42e/users/5150d70b99634426b9dcb220a7c7fea1/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.669533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f21c2-0baf-4469-a43f-97188d00dfdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6655fbbd6fbd4335b2147f0f066bf42e, user_id=5150d70b99634426b9dcb220a7c7fea1, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:59.672307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f21c2-0baf-4469-a43f-97188d00dfdc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:59.685896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1276/2556] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:04:59 2022] PUT /identity/v3/projects/6655fbbd6fbd4335b2147f0f066bf42e/users/5150d70b99634426b9dcb220a7c7fea1/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:04:59.697235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973a9fe-16a0-447e-8a28-662626a5595d None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.697531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973a9fe-16a0-447e-8a28-662626a5595d None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.697845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973a9fe-16a0-447e-8a28-662626a5595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.726324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b973a9fe-16a0-447e-8a28-662626a5595d None None] MFA Rules not processed for user `5150d70b99634426b9dcb220a7c7fea1`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:04:59.727576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b973a9fe-16a0-447e-8a28-662626a5595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:04:59.801283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1281/2557] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:04:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:04:59.806531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f21c2-0baf-4469-a43f-97188d00dfdc admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.875105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f21c2-0baf-4469-a43f-97188d00dfdc admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:04:59.881204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccaed9-b8ed-4d35-8f5e-b6aa98b8ef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:04:59.882001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccaed9-b8ed-4d35-8f5e-b6aa98b8ef6e admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:04:59.882371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccaed9-b8ed-4d35-8f5e-b6aa98b8ef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:04:59.882735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccaed9-b8ed-4d35-8f5e-b6aa98b8ef6e admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:04:59.883689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccaed9-b8ed-4d35-8f5e-b6aa98b8ef6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:04:59.886049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccaed9-b8ed-4d35-8f5e-b6aa98b8ef6e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:04:59.918174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1277/2558] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:04:59 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:04:59.922702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973a9fe-16a0-447e-8a28-662626a5595d None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:04:59.936251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973a9fe-16a0-447e-8a28-662626a5595d None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:00.003709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685c055-d2ea-4fc4-a056-951ef04ff1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:00.004393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685c055-d2ea-4fc4-a056-951ef04ff1eb admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:00.004659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685c055-d2ea-4fc4-a056-951ef04ff1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:00.004936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685c055-d2ea-4fc4-a056-951ef04ff1eb admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:00.005637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685c055-d2ea-4fc4-a056-951ef04ff1eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:00.007623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685c055-d2ea-4fc4-a056-951ef04ff1eb admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:00.026995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:00.036451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1282/2559] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:04:59 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:00.038289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccaed9-b8ed-4d35-8f5e-b6aa98b8ef6e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:00.053966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccaed9-b8ed-4d35-8f5e-b6aa98b8ef6e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:00.060305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c4b40-a5d2-4d21-986b-420723fd6e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:00.061243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4b40-a5d2-4d21-986b-420723fd6e78 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:00.062506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4b40-a5d2-4d21-986b-420723fd6e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:00.062506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4b40-a5d2-4d21-986b-420723fd6e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:00.063247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c4b40-a5d2-4d21-986b-420723fd6e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:00.064902 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c4b40-a5d2-4d21-986b-420723fd6e78 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:00.077126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1278/2560] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:00.082177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685c055-d2ea-4fc4-a056-951ef04ff1eb admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:00.097045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685c055-d2ea-4fc4-a056-951ef04ff1eb admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:00.103071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de82fc-67bc-402f-9cc1-b0b0b7761e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:00.103976 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de82fc-67bc-402f-9cc1-b0b0b7761e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:00.104256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de82fc-67bc-402f-9cc1-b0b0b7761e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:00.104531 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de82fc-67bc-402f-9cc1-b0b0b7761e2c admin admin] PATH_INFO: `/v3/projects/b71b3a37f77f4ec698afdb40680ec642/users/c3a7cdb37a7945e4a1af8e5010c2142c/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:00.121374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de82fc-67bc-402f-9cc1-b0b0b7761e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71b3a37f77f4ec698afdb40680ec642, user_id=c3a7cdb37a7945e4a1af8e5010c2142c, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:00.124485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de82fc-67bc-402f-9cc1-b0b0b7761e2c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:00.139867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1283/2561] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:00 2022] PUT /identity/v3/projects/b71b3a37f77f4ec698afdb40680ec642/users/c3a7cdb37a7945e4a1af8e5010c2142c/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:00.143921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c4b40-a5d2-4d21-986b-420723fd6e78 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:00.154506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c4b40-a5d2-4d21-986b-420723fd6e78 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:00.184253 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:00.229321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cc8bd-d062-4ef0-9c2b-ae2ac79632c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:00.230029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cc8bd-d062-4ef0-9c2b-ae2ac79632c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:00.230313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cc8bd-d062-4ef0-9c2b-ae2ac79632c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:00.230590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cc8bd-d062-4ef0-9c2b-ae2ac79632c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:00.231327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cc8bd-d062-4ef0-9c2b-ae2ac79632c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:00.232945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cc8bd-d062-4ef0-9c2b-ae2ac79632c6 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:00.243875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1279/2562] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:00.249194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de82fc-67bc-402f-9cc1-b0b0b7761e2c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:00.259965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de82fc-67bc-402f-9cc1-b0b0b7761e2c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:00.264927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:00.265985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:00.266264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:00.266535 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] PATH_INFO: `/v3/projects/b71b3a37f77f4ec698afdb40680ec642/users/c3a7cdb37a7945e4a1af8e5010c2142c/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:00.268569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71b3a37f77f4ec698afdb40680ec642, user_id=c3a7cdb37a7945e4a1af8e5010c2142c, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:00.271261 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:00.282215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1284/2563] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:00 2022] PUT /identity/v3/projects/b71b3a37f77f4ec698afdb40680ec642/users/c3a7cdb37a7945e4a1af8e5010c2142c/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:05:00.302202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be67e8-992c-4366-bc14-c4acfb96b9ad None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:00.302491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be67e8-992c-4366-bc14-c4acfb96b9ad None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:00.302768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be67e8-992c-4366-bc14-c4acfb96b9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:00.333408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-80be67e8-992c-4366-bc14-c4acfb96b9ad None None] MFA Rules not processed for user `c3a7cdb37a7945e4a1af8e5010c2142c`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:00.334670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-80be67e8-992c-4366-bc14-c4acfb96b9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:00.411956 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1280/2564] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:00.424652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:00.426858 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:00.491192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe71205-5562-4623-be6f-55ddfe548f27 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:00.495203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8419589-6708-45c5-b6de-768f3a40473b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:00.495853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8419589-6708-45c5-b6de-768f3a40473b service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:00.496125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8419589-6708-45c5-b6de-768f3a40473b service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:00.496404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8419589-6708-45c5-b6de-768f3a40473b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:00.506263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8419589-6708-45c5-b6de-768f3a40473b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:00.507669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8419589-6708-45c5-b6de-768f3a40473b service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:00.517980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1285/2565] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:00.602129 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be67e8-992c-4366-bc14-c4acfb96b9ad None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:00.613211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be67e8-992c-4366-bc14-c4acfb96b9ad None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:00.618100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659453f-6567-478e-92ab-928f823cb615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:00.618800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659453f-6567-478e-92ab-928f823cb615 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:00.619107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659453f-6567-478e-92ab-928f823cb615 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:00.619383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659453f-6567-478e-92ab-928f823cb615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:00.690723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659453f-6567-478e-92ab-928f823cb615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:00.692182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659453f-6567-478e-92ab-928f823cb615 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:00.701701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1281/2566] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:05.510406 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8419589-6708-45c5-b6de-768f3a40473b service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:05.578387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8419589-6708-45c5-b6de-768f3a40473b service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:05.583324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:05.584212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:05.584492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:05.584765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] PATH_INFO: `/v3/users/5150d70b99634426b9dcb220a7c7fea1` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:05.585996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5150d70b99634426b9dcb220a7c7fea1)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:05.587929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:05.643393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5150d70b99634426b9dcb220a7c7fea1'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:05.656194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5150d70b99634426b9dcb220a7c7fea1'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:05.658943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:05.664484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5150d70b99634426b9dcb220a7c7fea1'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:05.665470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5150d70b99634426b9dcb220a7c7fea1'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:05.676093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5150d70b99634426b9dcb220a7c7fea1'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:05.682512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1286/2567] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:05:05 2022] DELETE /identity/v3/users/5150d70b99634426b9dcb220a7c7fea1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:05.687516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c76b-76c7-40ea-b042-705cce2198d9 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:05.688371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c76b-76c7-40ea-b042-705cce2198d9 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:05.688371 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c76b-76c7-40ea-b042-705cce2198d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:05.706953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-26b0c76b-76c7-40ea-b042-705cce2198d9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:05.708186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-26b0c76b-76c7-40ea-b042-705cce2198d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:05.759238 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:05.775855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1282/2568] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:05:05.786030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:05.787757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:05.855597 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d973c9-c023-4570-948d-ac6a2ede615a admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:05.860073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d3c8e-257f-4689-870c-dc65f7a33819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:05.860764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d3c8e-257f-4689-870c-dc65f7a33819 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:05.861054 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d3c8e-257f-4689-870c-dc65f7a33819 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:05.861336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d3c8e-257f-4689-870c-dc65f7a33819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:05.871051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d3c8e-257f-4689-870c-dc65f7a33819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:05.872502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d3c8e-257f-4689-870c-dc65f7a33819 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:05.884231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1287/2569] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:06.158878 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0c76b-76c7-40ea-b042-705cce2198d9 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:06.160680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-26b0c76b-76c7-40ea-b042-705cce2198d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:06.184953 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0c76b-76c7-40ea-b042-705cce2198d9 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:06.192439 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:06.193223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:06.194282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:06.194574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] PATH_INFO: `/v3/projects/6655fbbd6fbd4335b2147f0f066bf42e` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:06.196366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6655fbbd6fbd4335b2147f0f066bf42e)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:06.198509 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:06.254286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6655fbbd6fbd4335b2147f0f066bf42e', 'request_id': 'req-f88a1861-d25a-46e5-b776-c9dd8f83f23f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.260638 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6655fbbd6fbd4335b2147f0f066bf42e', 'request_id': 'req-f88a1861-d25a-46e5-b776-c9dd8f83f23f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.269216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6655fbbd6fbd4335b2147f0f066bf42e', 'request_id': 'req-f88a1861-d25a-46e5-b776-c9dd8f83f23f'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.280424 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] The token cache is being invalidate because project 6655fbbd6fbd4335b2147f0f066bf42e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:05:06.280678 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.281351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.283013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1283/2570] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:05:06 2022] DELETE /identity/v3/projects/6655fbbd6fbd4335b2147f0f066bf42e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:06.287090 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d3c8e-257f-4689-870c-dc65f7a33819 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:06.297168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d3c8e-257f-4689-870c-dc65f7a33819 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:06.362034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:06.362945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:06.363219 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:06.363492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] PATH_INFO: `/v3/users/c3a7cdb37a7945e4a1af8e5010c2142c` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:06.364686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a7cdb37a7945e4a1af8e5010c2142c)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:06.366696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:06.430959 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a7cdb37a7945e4a1af8e5010c2142c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.443919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a7cdb37a7945e4a1af8e5010c2142c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.452680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a7cdb37a7945e4a1af8e5010c2142c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.453755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a7cdb37a7945e4a1af8e5010c2142c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.461118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-7a9254dc-82e6-42d5-8e35-d5e5ceb28ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a7cdb37a7945e4a1af8e5010c2142c'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.467325 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1288/2571] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:05:06 2022] DELETE /identity/v3/users/c3a7cdb37a7945e4a1af8e5010c2142c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:05:06.764605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:06.766805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:06.780623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a1861-d25a-46e5-b776-c9dd8f83f23f admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:06.847574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:06.848448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:06.848727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:06.848998 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] PATH_INFO: `/v3/projects/b71b3a37f77f4ec698afdb40680ec642` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:06.850272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b71b3a37f77f4ec698afdb40680ec642)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:06.852163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:06.903520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b71b3a37f77f4ec698afdb40680ec642', 'request_id': 'req-0e4c5994-53f0-4902-a624-dcefbef0752c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.910030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b71b3a37f77f4ec698afdb40680ec642', 'request_id': 'req-0e4c5994-53f0-4902-a624-dcefbef0752c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.919483 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b71b3a37f77f4ec698afdb40680ec642', 'request_id': 'req-0e4c5994-53f0-4902-a624-dcefbef0752c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.931197 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] The token cache is being invalidate because project b71b3a37f77f4ec698afdb40680ec642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:05:06.931484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.932136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0e4c5994-53f0-4902-a624-dcefbef0752c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:06.933744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1284/2572] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:05:06 2022] DELETE /identity/v3/projects/b71b3a37f77f4ec698afdb40680ec642 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:06.941017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9124d-9cd5-4004-bb45-cc61a6f173a1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:06.941571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9124d-9cd5-4004-bb45-cc61a6f173a1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:06.941686 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9124d-9cd5-4004-bb45-cc61a6f173a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:06.959494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:06.967965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-abf9124d-9cd5-4004-bb45-cc61a6f173a1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:06.970144 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-abf9124d-9cd5-4004-bb45-cc61a6f173a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:07.038257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1289/2573] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:07.047683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf30829-f204-47b9-9ec0-e0ca401fc78d None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.047872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf30829-f204-47b9-9ec0-e0ca401fc78d None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.048154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf30829-f204-47b9-9ec0-e0ca401fc78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.055186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:07.071058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-bbf30829-f204-47b9-9ec0-e0ca401fc78d None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:07.072262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-bbf30829-f204-47b9-9ec0-e0ca401fc78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:07.081951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1285/2574] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:07.086594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9124d-9cd5-4004-bb45-cc61a6f173a1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.098471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9124d-9cd5-4004-bb45-cc61a6f173a1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.103644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.104307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.104588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.104887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.105610 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:05:07.105949 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.107564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.120668 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1290/2575] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:05:07.125506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf30829-f204-47b9-9ec0-e0ca401fc78d None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.135514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf30829-f204-47b9-9ec0-e0ca401fc78d None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.141460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803917b-26a5-4c11-b2b9-68908e2cd878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.142239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803917b-26a5-4c11-b2b9-68908e2cd878 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.142524 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803917b-26a5-4c11-b2b9-68908e2cd878 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.142800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803917b-26a5-4c11-b2b9-68908e2cd878 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.143505 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803917b-26a5-4c11-b2b9-68908e2cd878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.145432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803917b-26a5-4c11-b2b9-68908e2cd878 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.175677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1286/2576] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:05:07 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:07.181727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.197326 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3910d-87cf-4461-b1ae-5c87684832cc admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.261981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd05bdb-fe64-470b-baaa-7935bc56f7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.262660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd05bdb-fe64-470b-baaa-7935bc56f7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.262940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd05bdb-fe64-470b-baaa-7935bc56f7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.263212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd05bdb-fe64-470b-baaa-7935bc56f7e3 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.263903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd05bdb-fe64-470b-baaa-7935bc56f7e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.265847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd05bdb-fe64-470b-baaa-7935bc56f7e3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.289608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1291/2577] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:05:07 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:07.293930 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803917b-26a5-4c11-b2b9-68908e2cd878 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.305528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803917b-26a5-4c11-b2b9-68908e2cd878 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.310995 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf3779-b0be-425e-b961-05352e1117b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.311887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf3779-b0be-425e-b961-05352e1117b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.312236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf3779-b0be-425e-b961-05352e1117b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.312688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf3779-b0be-425e-b961-05352e1117b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.313807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf3779-b0be-425e-b961-05352e1117b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.315529 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf3779-b0be-425e-b961-05352e1117b5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.326353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1287/2578] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:07.331460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd05bdb-fe64-470b-baaa-7935bc56f7e3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.343383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd05bdb-fe64-470b-baaa-7935bc56f7e3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.349263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac896bb-8c1b-4934-9f4b-8e1045ae86e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.349976 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac896bb-8c1b-4934-9f4b-8e1045ae86e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.350255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac896bb-8c1b-4934-9f4b-8e1045ae86e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.350528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac896bb-8c1b-4934-9f4b-8e1045ae86e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.351258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac896bb-8c1b-4934-9f4b-8e1045ae86e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.352984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac896bb-8c1b-4934-9f4b-8e1045ae86e0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.364368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1292/2579] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:07.368864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf3779-b0be-425e-b961-05352e1117b5 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.379203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf3779-b0be-425e-b961-05352e1117b5 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.384112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28accc84-1f96-40e7-b6ed-c1a1c1d15f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.385003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28accc84-1f96-40e7-b6ed-c1a1c1d15f00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.385284 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28accc84-1f96-40e7-b6ed-c1a1c1d15f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.385558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28accc84-1f96-40e7-b6ed-c1a1c1d15f00 admin admin] PATH_INFO: `/v3/projects/8f4ab5159cc4425b85ff7c56d6c9b5d8/users/3fe70657a3b34e06ad717e488004962c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.401799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28accc84-1f96-40e7-b6ed-c1a1c1d15f00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4ab5159cc4425b85ff7c56d6c9b5d8, user_id=3fe70657a3b34e06ad717e488004962c, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.404460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28accc84-1f96-40e7-b6ed-c1a1c1d15f00 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.419330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1288/2580] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:07 2022] PUT /identity/v3/projects/8f4ab5159cc4425b85ff7c56d6c9b5d8/users/3fe70657a3b34e06ad717e488004962c/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:07.430011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793a76-452c-4e2a-a515-7af80e67e6a6 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.430297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793a76-452c-4e2a-a515-7af80e67e6a6 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.430572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793a76-452c-4e2a-a515-7af80e67e6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.462870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-33793a76-452c-4e2a-a515-7af80e67e6a6 None None] MFA Rules not processed for user `3fe70657a3b34e06ad717e488004962c`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:07.464133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-33793a76-452c-4e2a-a515-7af80e67e6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:07.538979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1293/2581] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:07.543771 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28accc84-1f96-40e7-b6ed-c1a1c1d15f00 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.612943 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28accc84-1f96-40e7-b6ed-c1a1c1d15f00 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.618167 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b20c50-cace-4eb6-826a-65f98ac94ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.618834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20c50-cace-4eb6-826a-65f98ac94ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.619117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20c50-cace-4eb6-826a-65f98ac94ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.619391 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b20c50-cace-4eb6-826a-65f98ac94ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.620083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b20c50-cace-4eb6-826a-65f98ac94ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.622035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b20c50-cace-4eb6-826a-65f98ac94ca7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.652242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1289/2582] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:05:07 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:07.658019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793a76-452c-4e2a-a515-7af80e67e6a6 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.663523 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:07.670967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793a76-452c-4e2a-a515-7af80e67e6a6 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.735046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17365a-95fb-4601-b3a4-43956b713a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.735723 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17365a-95fb-4601-b3a4-43956b713a1c admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.736064 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17365a-95fb-4601-b3a4-43956b713a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.736259 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17365a-95fb-4601-b3a4-43956b713a1c admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.737057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17365a-95fb-4601-b3a4-43956b713a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.739312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17365a-95fb-4601-b3a4-43956b713a1c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.761888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1294/2583] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:05:07 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:07.767048 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b20c50-cace-4eb6-826a-65f98ac94ca7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.778231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b20c50-cace-4eb6-826a-65f98ac94ca7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.783365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eea34-4c87-4716-8221-8e76f12b2b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.784032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eea34-4c87-4716-8221-8e76f12b2b13 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.784317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eea34-4c87-4716-8221-8e76f12b2b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.784564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eea34-4c87-4716-8221-8e76f12b2b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.785290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eea34-4c87-4716-8221-8e76f12b2b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.786925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eea34-4c87-4716-8221-8e76f12b2b13 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.798272 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1290/2584] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:07.802695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17365a-95fb-4601-b3a4-43956b713a1c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.814498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17365a-95fb-4601-b3a4-43956b713a1c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.819623 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e077e-e5c2-4ba2-bf93-43aae9564cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.820299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e077e-e5c2-4ba2-bf93-43aae9564cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.820573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e077e-e5c2-4ba2-bf93-43aae9564cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.820846 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e077e-e5c2-4ba2-bf93-43aae9564cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.821583 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e077e-e5c2-4ba2-bf93-43aae9564cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.823236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e077e-e5c2-4ba2-bf93-43aae9564cb5 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.835631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1295/2585] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:07.840159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eea34-4c87-4716-8221-8e76f12b2b13 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:07.852214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eea34-4c87-4716-8221-8e76f12b2b13 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:07.856989 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d94fa-d8e1-41fc-ae45-a8080b9a485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:07.857944 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d94fa-d8e1-41fc-ae45-a8080b9a485e admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.858226 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d94fa-d8e1-41fc-ae45-a8080b9a485e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.858498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d94fa-d8e1-41fc-ae45-a8080b9a485e admin admin] PATH_INFO: `/v3/projects/27bb2be13ea1405f9c72bf0a302a8f38/users/c746ea0440d64c57a5beb00ae902aa20/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.875985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d94fa-d8e1-41fc-ae45-a8080b9a485e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bb2be13ea1405f9c72bf0a302a8f38, user_id=c746ea0440d64c57a5beb00ae902aa20, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:07.878715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d94fa-d8e1-41fc-ae45-a8080b9a485e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:07.893453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1291/2586] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:07 2022] PUT /identity/v3/projects/27bb2be13ea1405f9c72bf0a302a8f38/users/c746ea0440d64c57a5beb00ae902aa20/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:07.904422 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac86987-77ba-49a7-b936-124fb747b050 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:07.904755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac86987-77ba-49a7-b936-124fb747b050 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:07.905145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac86987-77ba-49a7-b936-124fb747b050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:07.938106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-aac86987-77ba-49a7-b936-124fb747b050 None None] MFA Rules not processed for user `c746ea0440d64c57a5beb00ae902aa20`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:07.939321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-aac86987-77ba-49a7-b936-124fb747b050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:08.007744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1296/2587] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:08.012551 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d94fa-d8e1-41fc-ae45-a8080b9a485e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:08.067894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:08.084106 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d94fa-d8e1-41fc-ae45-a8080b9a485e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:08.089231 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd6c2c-8d10-4792-a374-f24aa31e0bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:08.089937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd6c2c-8d10-4792-a374-f24aa31e0bcc admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:08.090216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd6c2c-8d10-4792-a374-f24aa31e0bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:08.090492 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd6c2c-8d10-4792-a374-f24aa31e0bcc admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:08.091194 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd6c2c-8d10-4792-a374-f24aa31e0bcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:08.093471 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd6c2c-8d10-4792-a374-f24aa31e0bcc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:08.120754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1292/2588] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:05:08 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:08.125493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac86987-77ba-49a7-b936-124fb747b050 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:08.137383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac86987-77ba-49a7-b936-124fb747b050 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:08.204083 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88fa3-91d3-44c5-a13f-ad59981d66a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:08.204930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88fa3-91d3-44c5-a13f-ad59981d66a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:08.205254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88fa3-91d3-44c5-a13f-ad59981d66a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:08.205515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc88fa3-91d3-44c5-a13f-ad59981d66a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:08.206402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc88fa3-91d3-44c5-a13f-ad59981d66a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:08.208423 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc88fa3-91d3-44c5-a13f-ad59981d66a7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:08.229655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:08.244017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1297/2589] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:05:08 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:08.249079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd6c2c-8d10-4792-a374-f24aa31e0bcc admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:08.261351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd6c2c-8d10-4792-a374-f24aa31e0bcc admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:08.268765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69c2d8-05fe-44de-9c2b-9353afcbac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:08.269462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c2d8-05fe-44de-9c2b-9353afcbac54 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:08.269809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c2d8-05fe-44de-9c2b-9353afcbac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:08.270184 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c2d8-05fe-44de-9c2b-9353afcbac54 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:08.270883 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69c2d8-05fe-44de-9c2b-9353afcbac54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:08.272658 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69c2d8-05fe-44de-9c2b-9353afcbac54 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:08.285220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1293/2590] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:05:08.291155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88fa3-91d3-44c5-a13f-ad59981d66a7 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:08.303051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc88fa3-91d3-44c5-a13f-ad59981d66a7 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:08.308067 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595b915-58ee-4dbc-be16-15abf75cd4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:08.309115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595b915-58ee-4dbc-be16-15abf75cd4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:08.309400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595b915-58ee-4dbc-be16-15abf75cd4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:08.309710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595b915-58ee-4dbc-be16-15abf75cd4b8 admin admin] PATH_INFO: `/v3/projects/fe1a9893ccc04869ad296cabffa5cff8/users/9e9b2bf603ff4e0784170a129fe143c2/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:08.326281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595b915-58ee-4dbc-be16-15abf75cd4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1a9893ccc04869ad296cabffa5cff8, user_id=9e9b2bf603ff4e0784170a129fe143c2, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:08.329763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595b915-58ee-4dbc-be16-15abf75cd4b8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:08.346743 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1298/2591] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:08 2022] PUT /identity/v3/projects/fe1a9893ccc04869ad296cabffa5cff8/users/9e9b2bf603ff4e0784170a129fe143c2/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:08.351103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69c2d8-05fe-44de-9c2b-9353afcbac54 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:08.362286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69c2d8-05fe-44de-9c2b-9353afcbac54 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:08.430049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f4224-557c-4ec6-8375-0b956c583e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:08.430634 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4224-557c-4ec6-8375-0b956c583e68 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:08.430912 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4224-557c-4ec6-8375-0b956c583e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:08.431186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f4224-557c-4ec6-8375-0b956c583e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:08.432085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f4224-557c-4ec6-8375-0b956c583e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:08.433705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f4224-557c-4ec6-8375-0b956c583e68 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:08.445746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1294/2592] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:05:08.450255 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595b915-58ee-4dbc-be16-15abf75cd4b8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:08.462515 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595b915-58ee-4dbc-be16-15abf75cd4b8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:08.468970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:08.469923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:08.470231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:08.470510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] PATH_INFO: `/v3/projects/fe1a9893ccc04869ad296cabffa5cff8/users/9e9b2bf603ff4e0784170a129fe143c2/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:08.472853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1a9893ccc04869ad296cabffa5cff8, user_id=9e9b2bf603ff4e0784170a129fe143c2, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:08.475775 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:08.487817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1299/2593] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:08 2022] PUT /identity/v3/projects/fe1a9893ccc04869ad296cabffa5cff8/users/9e9b2bf603ff4e0784170a129fe143c2/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:08.500732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf619c0-b4b0-404c-9893-8b9589981fcc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:08.501020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf619c0-b4b0-404c-9893-8b9589981fcc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:08.501301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf619c0-b4b0-404c-9893-8b9589981fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:08.535760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3cf619c0-b4b0-404c-9893-8b9589981fcc None None] MFA Rules not processed for user `9e9b2bf603ff4e0784170a129fe143c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:08.538705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3cf619c0-b4b0-404c-9893-8b9589981fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:08.633261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1295/2594] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:05:08.647087 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:08.649769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:08.715669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17103658-6b48-4ab0-82ed-da1f2a19d7f0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:08.720437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befadd1-05cf-4973-b2ec-6b2fea1b06a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:08.721098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befadd1-05cf-4973-b2ec-6b2fea1b06a8 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:08.721400 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befadd1-05cf-4973-b2ec-6b2fea1b06a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:08.721681 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befadd1-05cf-4973-b2ec-6b2fea1b06a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:08.783922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befadd1-05cf-4973-b2ec-6b2fea1b06a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:08.785408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9befadd1-05cf-4973-b2ec-6b2fea1b06a8 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:08.795229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1300/2595] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:10.324602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf619c0-b4b0-404c-9893-8b9589981fcc None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:10.326239 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3cf619c0-b4b0-404c-9893-8b9589981fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:10.339192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf619c0-b4b0-404c-9893-8b9589981fcc None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:10.343314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6d047-d2f1-475b-91a3-0fa300f33d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:10.343978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6d047-d2f1-475b-91a3-0fa300f33d89 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:10.344333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6d047-d2f1-475b-91a3-0fa300f33d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:10.344616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6d047-d2f1-475b-91a3-0fa300f33d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:10.393113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:10.410028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6d047-d2f1-475b-91a3-0fa300f33d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:10.411575 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6d047-d2f1-475b-91a3-0fa300f33d89 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:10.421537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1296/2596] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:10.976057 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befadd1-05cf-4973-b2ec-6b2fea1b06a8 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:10.985903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befadd1-05cf-4973-b2ec-6b2fea1b06a8 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:10.990989 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5597c1-436a-43ee-b23c-8a94073254c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:10.991671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5597c1-436a-43ee-b23c-8a94073254c1 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:10.991954 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5597c1-436a-43ee-b23c-8a94073254c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:10.992258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5597c1-436a-43ee-b23c-8a94073254c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:11.002286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5597c1-436a-43ee-b23c-8a94073254c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:11.003724 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5597c1-436a-43ee-b23c-8a94073254c1 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:11.013376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1301/2597] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:14.913387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6d047-d2f1-475b-91a3-0fa300f33d89 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:14.984703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6d047-d2f1-475b-91a3-0fa300f33d89 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:14.990632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:14.991518 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:14.991804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:14.992080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] PATH_INFO: `/v3/users/3fe70657a3b34e06ad717e488004962c` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:14.993313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe70657a3b34e06ad717e488004962c)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:14.995280 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:15.046011 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe70657a3b34e06ad717e488004962c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.062181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe70657a3b34e06ad717e488004962c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.071339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe70657a3b34e06ad717e488004962c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.079044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe70657a3b34e06ad717e488004962c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.079623 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe70657a3b34e06ad717e488004962c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.085836 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1297/2598] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:05:14 2022] DELETE /identity/v3/users/3fe70657a3b34e06ad717e488004962c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:15.091265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36384ed-6764-474d-b603-a55194ab74aa None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:15.091873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36384ed-6764-474d-b603-a55194ab74aa None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:15.091873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36384ed-6764-474d-b603-a55194ab74aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:15.111129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c36384ed-6764-474d-b603-a55194ab74aa None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:15.112322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c36384ed-6764-474d-b603-a55194ab74aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:15.179137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1302/2599] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:05:15.189138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:15.190710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:15.262940 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e39932-f87d-4da6-b114-c847dccd6035 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:15.267928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff525dd-eda2-4dc0-84f6-fa97fe704185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:15.268595 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff525dd-eda2-4dc0-84f6-fa97fe704185 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:15.268881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff525dd-eda2-4dc0-84f6-fa97fe704185 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:15.269152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff525dd-eda2-4dc0-84f6-fa97fe704185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:15.279304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff525dd-eda2-4dc0-84f6-fa97fe704185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:15.280785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff525dd-eda2-4dc0-84f6-fa97fe704185 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:15.290964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1298/2600] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:15.577066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36384ed-6764-474d-b603-a55194ab74aa None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:15.578938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c36384ed-6764-474d-b603-a55194ab74aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:15.589848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:15.595206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36384ed-6764-474d-b603-a55194ab74aa None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:15.603320 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:15.604512 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:15.604852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:15.605200 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] PATH_INFO: `/v3/projects/8f4ab5159cc4425b85ff7c56d6c9b5d8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:15.606442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f4ab5159cc4425b85ff7c56d6c9b5d8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:15.608368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:15.657460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f4ab5159cc4425b85ff7c56d6c9b5d8', 'request_id': 'req-3adbbf79-b35b-4551-ba21-2556e65005a0'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.666121 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f4ab5159cc4425b85ff7c56d6c9b5d8', 'request_id': 'req-3adbbf79-b35b-4551-ba21-2556e65005a0'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.672476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f4ab5159cc4425b85ff7c56d6c9b5d8', 'request_id': 'req-3adbbf79-b35b-4551-ba21-2556e65005a0'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.683611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] The token cache is being invalidate because project 8f4ab5159cc4425b85ff7c56d6c9b5d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:05:15.683984 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.684696 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.686425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1303/2601] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:05:15 2022] DELETE /identity/v3/projects/8f4ab5159cc4425b85ff7c56d6c9b5d8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:15.690839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff525dd-eda2-4dc0-84f6-fa97fe704185 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:15.697178 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:15.700896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff525dd-eda2-4dc0-84f6-fa97fe704185 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:15.760656 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:15.761546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:15.761858 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:15.762139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] PATH_INFO: `/v3/users/c746ea0440d64c57a5beb00ae902aa20` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:15.763341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c746ea0440d64c57a5beb00ae902aa20)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:15.765261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:15.816700 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c746ea0440d64c57a5beb00ae902aa20'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.832050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c746ea0440d64c57a5beb00ae902aa20'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.840584 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c746ea0440d64c57a5beb00ae902aa20'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.849138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c746ea0440d64c57a5beb00ae902aa20'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.849982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c746ea0440d64c57a5beb00ae902aa20'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:15.856950 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1299/2602] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:05:15 2022] DELETE /identity/v3/users/c746ea0440d64c57a5beb00ae902aa20 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:05:16.437111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:16.439125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:16.456035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbbf79-b35b-4551-ba21-2556e65005a0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:16.523233 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:16.524266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:16.524682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:16.525047 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] PATH_INFO: `/v3/projects/27bb2be13ea1405f9c72bf0a302a8f38` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:16.526493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bb2be13ea1405f9c72bf0a302a8f38)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:16.528485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:16.578739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bb2be13ea1405f9c72bf0a302a8f38', 'request_id': 'req-8d51e651-9188-4e0d-bbcd-50a2444aa342'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.586905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bb2be13ea1405f9c72bf0a302a8f38', 'request_id': 'req-8d51e651-9188-4e0d-bbcd-50a2444aa342'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.593590 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bb2be13ea1405f9c72bf0a302a8f38', 'request_id': 'req-8d51e651-9188-4e0d-bbcd-50a2444aa342'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.605444 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] The token cache is being invalidate because project 27bb2be13ea1405f9c72bf0a302a8f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:05:16.605847 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.606564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.608321 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1304/2603] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:05:16 2022] DELETE /identity/v3/projects/27bb2be13ea1405f9c72bf0a302a8f38 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:16.612416 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:16.614007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:16.628812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c7ae9-0c96-4206-a110-61191cf41ae9 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:16.686964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:16.687774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:16.688057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:16.688341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] PATH_INFO: `/v3/users/9e9b2bf603ff4e0784170a129fe143c2` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:16.690116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9b2bf603ff4e0784170a129fe143c2)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:16.692050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:16.724528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:16.753793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9b2bf603ff4e0784170a129fe143c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.767675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9b2bf603ff4e0784170a129fe143c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.777395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9b2bf603ff4e0784170a129fe143c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.786120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9b2bf603ff4e0784170a129fe143c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.786988 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-38c9c2a9-f0c6-4ce4-b650-c31519427e5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9b2bf603ff4e0784170a129fe143c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:16.794421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1300/2604] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:05:16 2022] DELETE /identity/v3/users/9e9b2bf603ff4e0784170a129fe143c2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:17.114307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:17.116126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:17.146378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51e651-9188-4e0d-bbcd-50a2444aa342 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:17.173946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:17.242587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:17.243488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.243768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.244039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] PATH_INFO: `/v3/projects/fe1a9893ccc04869ad296cabffa5cff8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.245265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1a9893ccc04869ad296cabffa5cff8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:17.250461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:17.311207 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1a9893ccc04869ad296cabffa5cff8', 'request_id': 'req-eb8f8a15-61da-4e70-a96e-bb59124525d6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:17.321408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1a9893ccc04869ad296cabffa5cff8', 'request_id': 'req-eb8f8a15-61da-4e70-a96e-bb59124525d6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:17.332347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1a9893ccc04869ad296cabffa5cff8', 'request_id': 'req-eb8f8a15-61da-4e70-a96e-bb59124525d6'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:17.346506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] The token cache is being invalidate because project fe1a9893ccc04869ad296cabffa5cff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:05:17.346795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:17.347533 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-eb8f8a15-61da-4e70-a96e-bb59124525d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:17.349413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1305/2605] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:05:17 2022] DELETE /identity/v3/projects/fe1a9893ccc04869ad296cabffa5cff8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:17.354532 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab127d-5b96-451f-bd37-9cf3120d52ca None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.354821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab127d-5b96-451f-bd37-9cf3120d52ca None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.355103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab127d-5b96-451f-bd37-9cf3120d52ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.375635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8cab127d-5b96-451f-bd37-9cf3120d52ca None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:17.377164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8cab127d-5b96-451f-bd37-9cf3120d52ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:17.447872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1301/2606] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:17.456341 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb96d9-b585-4cef-9bbd-18c6de967778 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.456869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb96d9-b585-4cef-9bbd-18c6de967778 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.456869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb96d9-b585-4cef-9bbd-18c6de967778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.476767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-d5fb96d9-b585-4cef-9bbd-18c6de967778 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:17.478048 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d5fb96d9-b585-4cef-9bbd-18c6de967778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:17.487006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1306/2607] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:17.491926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab127d-5b96-451f-bd37-9cf3120d52ca None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:17.504134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab127d-5b96-451f-bd37-9cf3120d52ca None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:17.509608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:17.510390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.510740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.511083 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.511910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:05:17.512288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:17.514017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:17.529498 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1302/2608] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:05:17.532780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb96d9-b585-4cef-9bbd-18c6de967778 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:17.542450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb96d9-b585-4cef-9bbd-18c6de967778 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:17.547586 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0dc06-9e3e-4239-81a1-7332cf6e9017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:17.548442 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0dc06-9e3e-4239-81a1-7332cf6e9017 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.548822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0dc06-9e3e-4239-81a1-7332cf6e9017 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.549153 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0dc06-9e3e-4239-81a1-7332cf6e9017 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.550143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0dc06-9e3e-4239-81a1-7332cf6e9017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:17.552238 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0dc06-9e3e-4239-81a1-7332cf6e9017 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:17.584004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1307/2609] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:05:17 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:05:17.585163 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:17.596911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7ecd-5145-4430-95d2-377e7db67dd4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:17.663918 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e61228-19f2-4b03-b1ad-8e475633de6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:17.664821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e61228-19f2-4b03-b1ad-8e475633de6a admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.665104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e61228-19f2-4b03-b1ad-8e475633de6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.665383 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e61228-19f2-4b03-b1ad-8e475633de6a admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.666123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e61228-19f2-4b03-b1ad-8e475633de6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:17.668092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e61228-19f2-4b03-b1ad-8e475633de6a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:17.690431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1303/2610] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:05:17 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:17.694695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0dc06-9e3e-4239-81a1-7332cf6e9017 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:17.705456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0dc06-9e3e-4239-81a1-7332cf6e9017 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:17.712190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9efe0-1a83-45d4-afe0-c42b8a737f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:17.712791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9efe0-1a83-45d4-afe0-c42b8a737f47 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.713074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9efe0-1a83-45d4-afe0-c42b8a737f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.713350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9efe0-1a83-45d4-afe0-c42b8a737f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.714120 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9efe0-1a83-45d4-afe0-c42b8a737f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:17.715769 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9efe0-1a83-45d4-afe0-c42b8a737f47 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:17.727301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1308/2611] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:17.731587 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e61228-19f2-4b03-b1ad-8e475633de6a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:17.745515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e61228-19f2-4b03-b1ad-8e475633de6a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:17.752185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723278e-6b12-49af-bcbe-d82b22bf1a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:17.752854 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723278e-6b12-49af-bcbe-d82b22bf1a51 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.753133 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723278e-6b12-49af-bcbe-d82b22bf1a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.753406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723278e-6b12-49af-bcbe-d82b22bf1a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.754200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723278e-6b12-49af-bcbe-d82b22bf1a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:17.755932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723278e-6b12-49af-bcbe-d82b22bf1a51 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:17.766979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1304/2612] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:17.771738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9efe0-1a83-45d4-afe0-c42b8a737f47 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:17.781906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9efe0-1a83-45d4-afe0-c42b8a737f47 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:17.787003 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:17.787891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.788175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.788451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] PATH_INFO: `/v3/projects/24a8e10ef5c94505af867ddf2edefe56/users/fac48f139f5d4e7b9895931aee78de75/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.802870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a8e10ef5c94505af867ddf2edefe56, user_id=fac48f139f5d4e7b9895931aee78de75, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:17.805527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:17.821882 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1309/2613] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:17 2022] PUT /identity/v3/projects/24a8e10ef5c94505af867ddf2edefe56/users/fac48f139f5d4e7b9895931aee78de75/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:17.833533 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7b4e-61df-4223-a1ee-f1c45e225ab2 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:17.833951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7b4e-61df-4223-a1ee-f1c45e225ab2 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:17.834233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf7b4e-61df-4223-a1ee-f1c45e225ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:17.852932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:17.865440 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-70cf7b4e-61df-4223-a1ee-f1c45e225ab2 None None] MFA Rules not processed for user `fac48f139f5d4e7b9895931aee78de75`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:17.867506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-70cf7b4e-61df-4223-a1ee-f1c45e225ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:17.943179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1305/2614] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:17.957619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:17.959710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:18.039312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:18.056459 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca9f2-83b5-493c-8353-d7715e18c435 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:18.061595 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cb5dc4c5a5a448294c57a2db5938d9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:18.063433 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:18.064766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:18.065093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:18.075621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:18.077380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:18.090174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1310/2615] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:17 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:21.146653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71853f71-e35d-4940-a46b-cdecdb792e8f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:21.147175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71853f71-e35d-4940-a46b-cdecdb792e8f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:21.147175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71853f71-e35d-4940-a46b-cdecdb792e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:21.166092 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-71853f71-e35d-4940-a46b-cdecdb792e8f None None] MFA Rules not processed for user `0943474b3d4d49d7973619388dab2be1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:21.167308 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-71853f71-e35d-4940-a46b-cdecdb792e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:21.228649 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1306/2616] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:21.881888 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:21.883501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:21.954252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5566cce-e66f-4d38-bd95-08e1fd95d2cc service glance] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:21.958032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:21.958688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:21.958975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:21.959249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:21.961031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:22.032376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:22.034150 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:22.045198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1311/2617] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:27.139214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71853f71-e35d-4940-a46b-cdecdb792e8f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:27.140830 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-71853f71-e35d-4940-a46b-cdecdb792e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:27.162873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71853f71-e35d-4940-a46b-cdecdb792e8f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:27.168038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:27.168946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:27.169233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:27.169517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] PATH_INFO: `/v3/users/75407a21b1fd46e8ba10540fc5a4c30a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:27.170768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=75407a21b1fd46e8ba10540fc5a4c30a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:27.172817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:27.226116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75407a21b1fd46e8ba10540fc5a4c30a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.243196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75407a21b1fd46e8ba10540fc5a4c30a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.252948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75407a21b1fd46e8ba10540fc5a4c30a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.260614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75407a21b1fd46e8ba10540fc5a4c30a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.261290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-95ad8654-0202-4692-a373-288b2a3407aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75407a21b1fd46e8ba10540fc5a4c30a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.267469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1307/2618] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:05:27 2022] DELETE /identity/v3/users/75407a21b1fd46e8ba10540fc5a4c30a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:27.463929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:27.476489 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c61cd-797b-4dca-bf49-a4d8849fbf3f service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:27.539873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:27.540755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:27.541037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:27.541308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] PATH_INFO: `/v3/projects/74f3f5c4b63c45d691e2b66883eda54a` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:27.542493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f3f5c4b63c45d691e2b66883eda54a)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:27.544428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:27.586690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:27.594737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f3f5c4b63c45d691e2b66883eda54a', 'request_id': 'req-f0a3beef-e632-444f-b961-38dd423c1390'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.605470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f3f5c4b63c45d691e2b66883eda54a', 'request_id': 'req-f0a3beef-e632-444f-b961-38dd423c1390'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.613137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f3f5c4b63c45d691e2b66883eda54a', 'request_id': 'req-f0a3beef-e632-444f-b961-38dd423c1390'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.625111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] The token cache is being invalidate because project 74f3f5c4b63c45d691e2b66883eda54a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:05:27.625415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.626089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-f0a3beef-e632-444f-b961-38dd423c1390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:05:27.628099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1312/2619] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:05:27 2022] DELETE /identity/v3/projects/74f3f5c4b63c45d691e2b66883eda54a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:27.633346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b480e40-6140-43ac-a2b0-c4979ea02945 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:27.633660 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b480e40-6140-43ac-a2b0-c4979ea02945 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:27.633979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b480e40-6140-43ac-a2b0-c4979ea02945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:27.657191 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-0b480e40-6140-43ac-a2b0-c4979ea02945 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:27.658460 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0b480e40-6140-43ac-a2b0-c4979ea02945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:27.670507 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:27.731075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1308/2620] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:05:27.740940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c18ca-d50d-440a-ba0f-2ea678abfa71 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:27.741232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c18ca-d50d-440a-ba0f-2ea678abfa71 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:27.741517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c18ca-d50d-440a-ba0f-2ea678abfa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:27.762198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-947c18ca-d50d-440a-ba0f-2ea678abfa71 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:27.763621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-947c18ca-d50d-440a-ba0f-2ea678abfa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:27.772214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1313/2621] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:27.776601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b480e40-6140-43ac-a2b0-c4979ea02945 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:27.787618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b480e40-6140-43ac-a2b0-c4979ea02945 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:27.793355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:27.794084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:27.794365 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:27.794704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:27.795463 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:05:27.795770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:27.797399 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:27.810531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1309/2622] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:05:27.815264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c18ca-d50d-440a-ba0f-2ea678abfa71 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:27.826190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c18ca-d50d-440a-ba0f-2ea678abfa71 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:27.832293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076adcae-fa39-4012-afd9-2f98913731fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:27.832937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076adcae-fa39-4012-afd9-2f98913731fb admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:27.833212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076adcae-fa39-4012-afd9-2f98913731fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:27.833490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076adcae-fa39-4012-afd9-2f98913731fb admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:27.834254 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076adcae-fa39-4012-afd9-2f98913731fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:27.836175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076adcae-fa39-4012-afd9-2f98913731fb admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:27.864672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1314/2623] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:05:27 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:27.870236 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:27.881437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81adba-18f0-438a-a85b-892ff86d8d13 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:27.946705 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f098df-0c96-4d9d-8a56-3f904fc2e494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:27.947462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f098df-0c96-4d9d-8a56-3f904fc2e494 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:27.947800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f098df-0c96-4d9d-8a56-3f904fc2e494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:27.948070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f098df-0c96-4d9d-8a56-3f904fc2e494 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:27.948781 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f098df-0c96-4d9d-8a56-3f904fc2e494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:27.950776 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f098df-0c96-4d9d-8a56-3f904fc2e494 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:27.977449 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1310/2624] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:05:27 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:27.981764 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076adcae-fa39-4012-afd9-2f98913731fb admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:27.993809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076adcae-fa39-4012-afd9-2f98913731fb admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:27.999902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9fa7-b1ab-42ef-ab8e-612b982d64a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.000656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9fa7-b1ab-42ef-ab8e-612b982d64a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.000917 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9fa7-b1ab-42ef-ab8e-612b982d64a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.001228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9fa7-b1ab-42ef-ab8e-612b982d64a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.002480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f9fa7-b1ab-42ef-ab8e-612b982d64a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.004719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f9fa7-b1ab-42ef-ab8e-612b982d64a0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.016742 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1315/2625] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:28.021765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f098df-0c96-4d9d-8a56-3f904fc2e494 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.032589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f098df-0c96-4d9d-8a56-3f904fc2e494 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.038535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304a124-99bb-45ff-b30c-12623e6e18e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.039270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304a124-99bb-45ff-b30c-12623e6e18e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.039556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304a124-99bb-45ff-b30c-12623e6e18e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.039834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304a124-99bb-45ff-b30c-12623e6e18e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.040572 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304a124-99bb-45ff-b30c-12623e6e18e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.042229 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304a124-99bb-45ff-b30c-12623e6e18e7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.054224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1311/2626] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:28.059443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9fa7-b1ab-42ef-ab8e-612b982d64a0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.076464 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9fa7-b1ab-42ef-ab8e-612b982d64a0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.083830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150b1d9-4f11-4152-8e36-1acc9bfb3118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.085106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150b1d9-4f11-4152-8e36-1acc9bfb3118 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.085548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150b1d9-4f11-4152-8e36-1acc9bfb3118 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.086035 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150b1d9-4f11-4152-8e36-1acc9bfb3118 admin admin] PATH_INFO: `/v3/projects/e608adfcfb4c4786ac1220411b0fa10d/users/3523cfc632b84b068751ca1a245de51e/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.103155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150b1d9-4f11-4152-8e36-1acc9bfb3118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e608adfcfb4c4786ac1220411b0fa10d, user_id=3523cfc632b84b068751ca1a245de51e, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.105914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150b1d9-4f11-4152-8e36-1acc9bfb3118 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.121419 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1316/2627] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:28 2022] PUT /identity/v3/projects/e608adfcfb4c4786ac1220411b0fa10d/users/3523cfc632b84b068751ca1a245de51e/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:05:28.132726 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460d2f6-46ab-4ce4-a024-0228729ae360 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.132994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460d2f6-46ab-4ce4-a024-0228729ae360 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.133351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460d2f6-46ab-4ce4-a024-0228729ae360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.169318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3460d2f6-46ab-4ce4-a024-0228729ae360 None None] MFA Rules not processed for user `3523cfc632b84b068751ca1a245de51e`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:28.171376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3460d2f6-46ab-4ce4-a024-0228729ae360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:28.250355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1312/2628] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:28.254848 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150b1d9-4f11-4152-8e36-1acc9bfb3118 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.327890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150b1d9-4f11-4152-8e36-1acc9bfb3118 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.332939 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36e78b-f5fa-47a1-80f6-e87fe93c11f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.333630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e78b-f5fa-47a1-80f6-e87fe93c11f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.333953 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e78b-f5fa-47a1-80f6-e87fe93c11f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.334276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e78b-f5fa-47a1-80f6-e87fe93c11f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.335018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36e78b-f5fa-47a1-80f6-e87fe93c11f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.337427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36e78b-f5fa-47a1-80f6-e87fe93c11f2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.376578 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1317/2629] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:05:28 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:28.378279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460d2f6-46ab-4ce4-a024-0228729ae360 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.389806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460d2f6-46ab-4ce4-a024-0228729ae360 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.402319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:28.455510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9c9b-15d0-4db7-83bf-680389dceb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.456152 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9c9b-15d0-4db7-83bf-680389dceb87 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.456635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9c9b-15d0-4db7-83bf-680389dceb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.456974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe9c9b-15d0-4db7-83bf-680389dceb87 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.457741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9c9b-15d0-4db7-83bf-680389dceb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.460001 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe9c9b-15d0-4db7-83bf-680389dceb87 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.487832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1313/2630] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:05:28 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:05:28.491765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36e78b-f5fa-47a1-80f6-e87fe93c11f2 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.502745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36e78b-f5fa-47a1-80f6-e87fe93c11f2 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.507749 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d153c4-6748-4083-bf47-baeb25d78010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.508435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d153c4-6748-4083-bf47-baeb25d78010 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.508726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d153c4-6748-4083-bf47-baeb25d78010 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.509001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d153c4-6748-4083-bf47-baeb25d78010 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.509795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d153c4-6748-4083-bf47-baeb25d78010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.511429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d153c4-6748-4083-bf47-baeb25d78010 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.523236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1318/2631] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:28.527741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9c9b-15d0-4db7-83bf-680389dceb87 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.548674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe9c9b-15d0-4db7-83bf-680389dceb87 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.554426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8faf0-8903-4ce9-9386-5154961eca25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.555353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8faf0-8903-4ce9-9386-5154961eca25 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.555641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8faf0-8903-4ce9-9386-5154961eca25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.555910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8faf0-8903-4ce9-9386-5154961eca25 admin admin] PATH_INFO: `/v3/projects/ae2fa1a6691f4a069c375062eb7e6a84/users/02cd229cba2e49a4807d07f2876c6be7/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.574467 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8faf0-8903-4ce9-9386-5154961eca25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2fa1a6691f4a069c375062eb7e6a84, user_id=02cd229cba2e49a4807d07f2876c6be7, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.577235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8faf0-8903-4ce9-9386-5154961eca25 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.592217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1314/2632] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:28 2022] PUT /identity/v3/projects/ae2fa1a6691f4a069c375062eb7e6a84/users/02cd229cba2e49a4807d07f2876c6be7/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:28.596838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d153c4-6748-4083-bf47-baeb25d78010 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.607690 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d153c4-6748-4083-bf47-baeb25d78010 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.645947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:28.674329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d64a1-2487-44cf-a368-25745105e0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.675223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d64a1-2487-44cf-a368-25745105e0ef admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.675534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d64a1-2487-44cf-a368-25745105e0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.675810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d64a1-2487-44cf-a368-25745105e0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.676576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d64a1-2487-44cf-a368-25745105e0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.678314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d64a1-2487-44cf-a368-25745105e0ef admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.689217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1319/2633] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:05:28.693822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8faf0-8903-4ce9-9386-5154961eca25 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.704514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8faf0-8903-4ce9-9386-5154961eca25 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.709680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.710567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.710849 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.711154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] PATH_INFO: `/v3/projects/ae2fa1a6691f4a069c375062eb7e6a84/users/02cd229cba2e49a4807d07f2876c6be7/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.713305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae2fa1a6691f4a069c375062eb7e6a84, user_id=02cd229cba2e49a4807d07f2876c6be7, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.716079 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.728774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1315/2634] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:05:28 2022] PUT /identity/v3/projects/ae2fa1a6691f4a069c375062eb7e6a84/users/02cd229cba2e49a4807d07f2876c6be7/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:05:28.739718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30ed09-e3b7-4fd1-9308-f2f0dfd9b9d5 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.739995 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30ed09-e3b7-4fd1-9308-f2f0dfd9b9d5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.740275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30ed09-e3b7-4fd1-9308-f2f0dfd9b9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.771107 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2d30ed09-e3b7-4fd1-9308-f2f0dfd9b9d5 None None] MFA Rules not processed for user `02cd229cba2e49a4807d07f2876c6be7`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:05:28.772346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2d30ed09-e3b7-4fd1-9308-f2f0dfd9b9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:28.843647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1320/2635] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:05:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:05:28.855598 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:28.857514 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:28.920007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5fe43-af8a-42c1-a7c9-8e1443e7faea admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:28.924468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:28.925221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:28.925886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:28.925886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:28.935708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:28.937262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:28.947810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1316/2636] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:29.024483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30ed09-e3b7-4fd1-9308-f2f0dfd9b9d5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:29.035051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30ed09-e3b7-4fd1-9308-f2f0dfd9b9d5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:29.039972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a370d-de65-4769-8377-00d675bbf329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:29.040943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a370d-de65-4769-8377-00d675bbf329 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:29.041257 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a370d-de65-4769-8377-00d675bbf329 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:29.041566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a370d-de65-4769-8377-00d675bbf329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:29.106879 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a370d-de65-4769-8377-00d675bbf329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:29.108324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a370d-de65-4769-8377-00d675bbf329 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:29.118206 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1321/2637] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:38.943787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=65,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:39.113828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=74 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:41.475003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:41.476438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:41.556631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fb46ca-4334-4e01-928b-e2000c87140c service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:41.562331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:41.562997 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:41.563265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:41.563563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:41.565384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:41.646777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a370d-de65-4769-8377-00d675bbf329 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:41.656103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a370d-de65-4769-8377-00d675bbf329 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:41.658915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:41.660728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:41.661307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:41.662065 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:41.662500 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:41.662728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:41.673627 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:41.675501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:41.680214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1317/2638] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:05:41.688243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1322/2639] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:05:51.677304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=69 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:51.682265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=6 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:05:58.705811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:05:58.707408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:58.838494 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df0f7f-fe2a-4ff7-98fb-2cead2169c3a service nova] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:05:58.842408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:05:58.843058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:05:58.843339 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:05:58.843616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:05:58.845469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:05:58.959175 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:05:58.960747 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:05:58.983030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1318/2640] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:01.948840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:01.950655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:02.019333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:02.020920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:02.035319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026d0e9-aed0-4c11-acaa-cdd20233b2eb service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.040522 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.041485 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.041862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.042242 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] PATH_INFO: `/v3/users/9a67a5c4b7034d22b78331b907cc20bc` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.043499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a67a5c4b7034d22b78331b907cc20bc)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.045418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.051775 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=100 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:02.064682 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba49f7-2675-492f-b916-b0cec6cc84d0 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.070217 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.070849 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.071132 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.071407 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.073374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:02.090408 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.092167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.098528 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a67a5c4b7034d22b78331b907cc20bc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.103315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1323/2641] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:02.111217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a67a5c4b7034d22b78331b907cc20bc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.120665 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a67a5c4b7034d22b78331b907cc20bc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.128468 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a67a5c4b7034d22b78331b907cc20bc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.129126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-ae14152e-9f30-412a-922f-fc8ec879f295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a67a5c4b7034d22b78331b907cc20bc'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.135421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1319/2642] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:02 2022] DELETE /identity/v3/users/9a67a5c4b7034d22b78331b907cc20bc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:02.316643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:02.328543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16860899-4b95-45b1-9f20-64275a0fa2dd service placement] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.387052 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.387925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.388204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.388475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] PATH_INFO: `/v3/projects/d20d29454c00499282eb86547eeb4c10` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.389688 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d20d29454c00499282eb86547eeb4c10)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.391613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.444055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd20d29454c00499282eb86547eeb4c10', 'request_id': 'req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.454538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd20d29454c00499282eb86547eeb4c10', 'request_id': 'req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.461328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd20d29454c00499282eb86547eeb4c10', 'request_id': 'req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.466168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:02.475313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] The token cache is being invalidate because project d20d29454c00499282eb86547eeb4c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:02.476016 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.477190 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-e2d6a1df-113d-40af-b0a9-b1650d60c2ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:02.479214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1324/2643] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:02 2022] DELETE /identity/v3/projects/d20d29454c00499282eb86547eeb4c10 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:02.484418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0e26-01c8-415e-b2a0-de22a40014a8 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.484760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0e26-01c8-415e-b2a0-de22a40014a8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.485044 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0e26-01c8-415e-b2a0-de22a40014a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.513564 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-e1ab0e26-01c8-415e-b2a0-de22a40014a8 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:02.514921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-e1ab0e26-01c8-415e-b2a0-de22a40014a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:02.587268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1320/2644] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:02.595171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faa585-7a69-42f0-bde9-a286dd4440f0 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.595461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faa585-7a69-42f0-bde9-a286dd4440f0 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.595738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faa585-7a69-42f0-bde9-a286dd4440f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.619136 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-24faa585-7a69-42f0-bde9-a286dd4440f0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:02.620536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-24faa585-7a69-42f0-bde9-a286dd4440f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:02.631044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1325/2645] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:02.635095 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0e26-01c8-415e-b2a0-de22a40014a8 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:02.645332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0e26-01c8-415e-b2a0-de22a40014a8 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.650787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.651477 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.651758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.652425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.652751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:06:02.653057 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.654698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.667088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1321/2646] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:06:02.671418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faa585-7a69-42f0-bde9-a286dd4440f0 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:02.681165 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faa585-7a69-42f0-bde9-a286dd4440f0 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.685765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d1ef9-f45e-40b1-b952-f3513129ec01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.686449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d1ef9-f45e-40b1-b952-f3513129ec01 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.686729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d1ef9-f45e-40b1-b952-f3513129ec01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.687003 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d1ef9-f45e-40b1-b952-f3513129ec01 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.687711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d1ef9-f45e-40b1-b952-f3513129ec01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.689628 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d1ef9-f45e-40b1-b952-f3513129ec01 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.717806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1326/2647] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:02 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:02.721222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:02.732005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfbb6c-346d-48e5-983a-1414767bfbe2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.792041 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a63537-38cd-42b9-8850-282204354ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.792749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a63537-38cd-42b9-8850-282204354ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.793047 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a63537-38cd-42b9-8850-282204354ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.793322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a63537-38cd-42b9-8850-282204354ec0 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.794071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a63537-38cd-42b9-8850-282204354ec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.796040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a63537-38cd-42b9-8850-282204354ec0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.806812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:02.822885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1322/2648] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:02 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:02.824174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d1ef9-f45e-40b1-b952-f3513129ec01 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:02.835309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d1ef9-f45e-40b1-b952-f3513129ec01 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.841284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0834c-75f0-4ee2-bf95-3a3f00894dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.841929 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0834c-75f0-4ee2-bf95-3a3f00894dca admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.842212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0834c-75f0-4ee2-bf95-3a3f00894dca admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.842482 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0834c-75f0-4ee2-bf95-3a3f00894dca admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.843223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0834c-75f0-4ee2-bf95-3a3f00894dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.844905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0834c-75f0-4ee2-bf95-3a3f00894dca admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.857011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1327/2649] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:06:02.861732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a63537-38cd-42b9-8850-282204354ec0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:02.877895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a63537-38cd-42b9-8850-282204354ec0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.884601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e015768-d030-4743-923c-dac7a3f2e366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.885264 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e015768-d030-4743-923c-dac7a3f2e366 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.885549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e015768-d030-4743-923c-dac7a3f2e366 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.885857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e015768-d030-4743-923c-dac7a3f2e366 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.886591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e015768-d030-4743-923c-dac7a3f2e366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.888183 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e015768-d030-4743-923c-dac7a3f2e366 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.898820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1323/2650] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:02.903576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0834c-75f0-4ee2-bf95-3a3f00894dca admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:02.914711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0834c-75f0-4ee2-bf95-3a3f00894dca admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:02.919494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461037-ac89-46bb-9c71-7219c410f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:02.920413 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461037-ac89-46bb-9c71-7219c410f17e admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.920692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461037-ac89-46bb-9c71-7219c410f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.920964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00461037-ac89-46bb-9c71-7219c410f17e admin admin] PATH_INFO: `/v3/projects/f513a8c071a84789b3de00ce0ca6b923/users/2745b8d273f34c8a8befd3340db055da/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.936808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00461037-ac89-46bb-9c71-7219c410f17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f513a8c071a84789b3de00ce0ca6b923, user_id=2745b8d273f34c8a8befd3340db055da, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:02.939557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00461037-ac89-46bb-9c71-7219c410f17e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:02.956553 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1328/2651] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:02 2022] PUT /identity/v3/projects/f513a8c071a84789b3de00ce0ca6b923/users/2745b8d273f34c8a8befd3340db055da/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:02.967216 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09b66-8573-453a-abc8-7ff42ee951c8 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:02.967508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09b66-8573-453a-abc8-7ff42ee951c8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:02.967784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09b66-8573-453a-abc8-7ff42ee951c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:02.997950 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8ae09b66-8573-453a-abc8-7ff42ee951c8 None None] MFA Rules not processed for user `2745b8d273f34c8a8befd3340db055da`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:02.999213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8ae09b66-8573-453a-abc8-7ff42ee951c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:03.065062 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1324/2652] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:03.071448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461037-ac89-46bb-9c71-7219c410f17e admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:03.136781 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00461037-ac89-46bb-9c71-7219c410f17e admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:03.142446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea83b7-ddb9-4cbd-a6db-568d99de3ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:03.143122 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea83b7-ddb9-4cbd-a6db-568d99de3ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.143403 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea83b7-ddb9-4cbd-a6db-568d99de3ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.143672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea83b7-ddb9-4cbd-a6db-568d99de3ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.144380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea83b7-ddb9-4cbd-a6db-568d99de3ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:03.146499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea83b7-ddb9-4cbd-a6db-568d99de3ab1 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:03.172484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1329/2653] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:03 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:03.176924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09b66-8573-453a-abc8-7ff42ee951c8 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:03.188050 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09b66-8573-453a-abc8-7ff42ee951c8 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:03.251566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a5bb7-d055-49c1-8d5a-c6153c4f0f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:03.252502 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a5bb7-d055-49c1-8d5a-c6153c4f0f99 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.252864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a5bb7-d055-49c1-8d5a-c6153c4f0f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.253209 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a5bb7-d055-49c1-8d5a-c6153c4f0f99 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.254262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a5bb7-d055-49c1-8d5a-c6153c4f0f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:03.257248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a5bb7-d055-49c1-8d5a-c6153c4f0f99 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:03.285228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1325/2654] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:03 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:06:03.290307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea83b7-ddb9-4cbd-a6db-568d99de3ab1 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:03.301754 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea83b7-ddb9-4cbd-a6db-568d99de3ab1 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:03.307699 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251fb79-5d16-4b80-99ef-e82091fd5150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:03.308371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251fb79-5d16-4b80-99ef-e82091fd5150 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.308656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251fb79-5d16-4b80-99ef-e82091fd5150 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.308932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251fb79-5d16-4b80-99ef-e82091fd5150 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.309670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251fb79-5d16-4b80-99ef-e82091fd5150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:03.311340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251fb79-5d16-4b80-99ef-e82091fd5150 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:03.322806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1330/2655] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:06:03.327616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a5bb7-d055-49c1-8d5a-c6153c4f0f99 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:03.343789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a5bb7-d055-49c1-8d5a-c6153c4f0f99 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:03.351418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63f527-bfdc-4bcc-8445-04d18797a801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:03.352326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f527-bfdc-4bcc-8445-04d18797a801 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.352616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f527-bfdc-4bcc-8445-04d18797a801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.352879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f527-bfdc-4bcc-8445-04d18797a801 admin admin] PATH_INFO: `/v3/projects/49b7b20fd3d24abd8272250814fbf87f/users/57a24160ea96405bacef6f770ede6acb/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.373734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63f527-bfdc-4bcc-8445-04d18797a801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b7b20fd3d24abd8272250814fbf87f, user_id=57a24160ea96405bacef6f770ede6acb, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:03.376431 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63f527-bfdc-4bcc-8445-04d18797a801 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:03.391727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1326/2656] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:03 2022] PUT /identity/v3/projects/49b7b20fd3d24abd8272250814fbf87f/users/57a24160ea96405bacef6f770ede6acb/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:03.395731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251fb79-5d16-4b80-99ef-e82091fd5150 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:03.406631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251fb79-5d16-4b80-99ef-e82091fd5150 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:03.480338 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353b3c8-9a68-496f-90fd-6e46eb84268b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:03.480808 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353b3c8-9a68-496f-90fd-6e46eb84268b admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.481270 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353b3c8-9a68-496f-90fd-6e46eb84268b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.481665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353b3c8-9a68-496f-90fd-6e46eb84268b admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.482799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353b3c8-9a68-496f-90fd-6e46eb84268b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:03.485198 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353b3c8-9a68-496f-90fd-6e46eb84268b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:03.498194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1331/2657] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:06:03.502756 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63f527-bfdc-4bcc-8445-04d18797a801 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:03.514000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63f527-bfdc-4bcc-8445-04d18797a801 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:03.519120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:03.520007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.520285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.520556 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] PATH_INFO: `/v3/projects/49b7b20fd3d24abd8272250814fbf87f/users/57a24160ea96405bacef6f770ede6acb/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.522976 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b7b20fd3d24abd8272250814fbf87f, user_id=57a24160ea96405bacef6f770ede6acb, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:03.525589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:03.536376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1327/2658] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:03 2022] PUT /identity/v3/projects/49b7b20fd3d24abd8272250814fbf87f/users/57a24160ea96405bacef6f770ede6acb/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:03.547542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d935-ea59-471e-b98f-4529c071bb9b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.547834 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d935-ea59-471e-b98f-4529c071bb9b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.548111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d935-ea59-471e-b98f-4529c071bb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.554486 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:03.578263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-31e4d935-ea59-471e-b98f-4529c071bb9b None None] MFA Rules not processed for user `57a24160ea96405bacef6f770ede6acb`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:03.579521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-31e4d935-ea59-471e-b98f-4529c071bb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:03.649376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1332/2659] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:03.661309 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:03.663576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:03.697757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:03.726820 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4b608-5031-4cff-bf3c-94fcc8722ac2 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:03.731453 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49a9f5-a411-453f-ba9e-4899fa55390d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:03.732108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a9f5-a411-453f-ba9e-4899fa55390d service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.732390 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a9f5-a411-453f-ba9e-4899fa55390d service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.732661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49a9f5-a411-453f-ba9e-4899fa55390d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.742993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49a9f5-a411-453f-ba9e-4899fa55390d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:03.744448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49a9f5-a411-453f-ba9e-4899fa55390d service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:03.754090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1328/2660] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:03.830503 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4d935-ea59-471e-b98f-4529c071bb9b None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:03.844155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4d935-ea59-471e-b98f-4529c071bb9b None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:03.850271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ce098-6355-40e9-ad5f-f4cca72f945f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:03.850924 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ce098-6355-40e9-ad5f-f4cca72f945f service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:03.851212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ce098-6355-40e9-ad5f-f4cca72f945f service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:03.851485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ce098-6355-40e9-ad5f-f4cca72f945f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:03.918154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ce098-6355-40e9-ad5f-f4cca72f945f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:03.919557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ce098-6355-40e9-ad5f-f4cca72f945f service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:03.931281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1333/2661] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:13.749892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=14 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:13.925943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=61 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:20.385571 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49a9f5-a411-453f-ba9e-4899fa55390d service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:20.448331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49a9f5-a411-453f-ba9e-4899fa55390d service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:20.454760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:20.455826 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:20.456214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:20.456592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] PATH_INFO: `/v3/users/2745b8d273f34c8a8befd3340db055da` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:20.457999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2745b8d273f34c8a8befd3340db055da)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:20.460112 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:20.515994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2745b8d273f34c8a8befd3340db055da'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:20.532132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2745b8d273f34c8a8befd3340db055da'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:20.539691 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2745b8d273f34c8a8befd3340db055da'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:20.546739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2745b8d273f34c8a8befd3340db055da'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:20.547438 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2745b8d273f34c8a8befd3340db055da'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:20.553585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1329/2662] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:20 2022] DELETE /identity/v3/users/2745b8d273f34c8a8befd3340db055da => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:20.558274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd164c46-b168-4dca-8d31-8eba5931343f None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:20.558780 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd164c46-b168-4dca-8d31-8eba5931343f None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:20.558887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd164c46-b168-4dca-8d31-8eba5931343f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:20.579791 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-dd164c46-b168-4dca-8d31-8eba5931343f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:20.581009 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-dd164c46-b168-4dca-8d31-8eba5931343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:20.645521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1334/2663] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:20.655448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:20.657287 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:20.722250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8990962-d44d-4976-bb26-845a2a245bfe admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:20.727632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41892d9-28db-4a2f-92d0-96ae015c3676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:20.727632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41892d9-28db-4a2f-92d0-96ae015c3676 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:20.728007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41892d9-28db-4a2f-92d0-96ae015c3676 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:20.728104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41892d9-28db-4a2f-92d0-96ae015c3676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:20.739182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41892d9-28db-4a2f-92d0-96ae015c3676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:20.740738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41892d9-28db-4a2f-92d0-96ae015c3676 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:20.751049 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1330/2664] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:21.047733 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd164c46-b168-4dca-8d31-8eba5931343f None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:21.049425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-dd164c46-b168-4dca-8d31-8eba5931343f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:21.062205 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd164c46-b168-4dca-8d31-8eba5931343f None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:21.067279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:21.068290 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.069131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.069131 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] PATH_INFO: `/v3/projects/f513a8c071a84789b3de00ce0ca6b923` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.070409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f513a8c071a84789b3de00ce0ca6b923)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:21.072435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:21.124287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f513a8c071a84789b3de00ce0ca6b923', 'request_id': 'req-261cba67-54be-4e54-ab87-51acd2377f81'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.132350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f513a8c071a84789b3de00ce0ca6b923', 'request_id': 'req-261cba67-54be-4e54-ab87-51acd2377f81'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.139587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f513a8c071a84789b3de00ce0ca6b923', 'request_id': 'req-261cba67-54be-4e54-ab87-51acd2377f81'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.151456 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] The token cache is being invalidate because project f513a8c071a84789b3de00ce0ca6b923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:21.151642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.152242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.153853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1335/2665] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:21 2022] DELETE /identity/v3/projects/f513a8c071a84789b3de00ce0ca6b923 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:21.158710 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41892d9-28db-4a2f-92d0-96ae015c3676 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:21.168699 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41892d9-28db-4a2f-92d0-96ae015c3676 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:21.188901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:21.231474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:21.232332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.232611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.232885 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] PATH_INFO: `/v3/users/57a24160ea96405bacef6f770ede6acb` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.234745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57a24160ea96405bacef6f770ede6acb)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:21.236283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:21.289111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57a24160ea96405bacef6f770ede6acb'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.301530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57a24160ea96405bacef6f770ede6acb'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.311055 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57a24160ea96405bacef6f770ede6acb'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.320384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57a24160ea96405bacef6f770ede6acb'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.321118 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-2fcd967f-9727-4568-8b6a-152e456675e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57a24160ea96405bacef6f770ede6acb'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.328404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1331/2666] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:21 2022] DELETE /identity/v3/users/57a24160ea96405bacef6f770ede6acb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:21.344655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:21.346438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:21.360503 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261cba67-54be-4e54-ab87-51acd2377f81 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:21.422453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:21.423352 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.423634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.423934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] PATH_INFO: `/v3/users/fac48f139f5d4e7b9895931aee78de75` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.425163 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac48f139f5d4e7b9895931aee78de75)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:21.427179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:21.457816 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:21.482074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac48f139f5d4e7b9895931aee78de75'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.496784 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac48f139f5d4e7b9895931aee78de75'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.505457 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac48f139f5d4e7b9895931aee78de75'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.506485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac48f139f5d4e7b9895931aee78de75'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.514675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac48f139f5d4e7b9895931aee78de75'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.521494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1336/2667] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:21 2022] DELETE /identity/v3/users/fac48f139f5d4e7b9895931aee78de75 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:21.526418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbc70c-001d-4b91-b6d9-34e0de18fb08 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.527020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbc70c-001d-4b91-b6d9-34e0de18fb08 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.527020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbc70c-001d-4b91-b6d9-34e0de18fb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.544770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1cbbc70c-001d-4b91-b6d9-34e0de18fb08 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:21.546125 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1cbbc70c-001d-4b91-b6d9-34e0de18fb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:21.613154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:21.616920 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1332/2668] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:06:21.617059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:21.623834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbc70c-001d-4b91-b6d9-34e0de18fb08 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:21.628500 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1cbbc70c-001d-4b91-b6d9-34e0de18fb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:21.630300 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359164e-9d0d-4f4a-9cd1-d9c5fbfde97f admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:21.644024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:21.644960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.645269 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.645520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] PATH_INFO: `/v3/projects/49b7b20fd3d24abd8272250814fbf87f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.647242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b7b20fd3d24abd8272250814fbf87f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:21.649213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:21.680205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:21.696683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbc70c-001d-4b91-b6d9-34e0de18fb08 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:21.702053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b7b20fd3d24abd8272250814fbf87f', 'request_id': 'req-df55a031-6d28-4933-b9ba-a75b2fac07f7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.710889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b7b20fd3d24abd8272250814fbf87f', 'request_id': 'req-df55a031-6d28-4933-b9ba-a75b2fac07f7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.717451 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b7b20fd3d24abd8272250814fbf87f', 'request_id': 'req-df55a031-6d28-4933-b9ba-a75b2fac07f7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.728971 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] The token cache is being invalidate because project 49b7b20fd3d24abd8272250814fbf87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:21.729252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.729950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-df55a031-6d28-4933-b9ba-a75b2fac07f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:21.731862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1337/2669] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:21 2022] DELETE /identity/v3/projects/49b7b20fd3d24abd8272250814fbf87f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:21.737313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea91ab5-553c-4f64-8863-0fefa64551ed None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.737593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea91ab5-553c-4f64-8863-0fefa64551ed None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.737891 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea91ab5-553c-4f64-8863-0fefa64551ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.755602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253f7a4-e106-4b1b-bf4a-f238bfea4d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:21.756298 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253f7a4-e106-4b1b-bf4a-f238bfea4d3e service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.756585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253f7a4-e106-4b1b-bf4a-f238bfea4d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.756862 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253f7a4-e106-4b1b-bf4a-f238bfea4d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.758832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2253f7a4-e106-4b1b-bf4a-f238bfea4d3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:21.787392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2ea91ab5-553c-4f64-8863-0fefa64551ed None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:21.788599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2ea91ab5-553c-4f64-8863-0fefa64551ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:21.823392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253f7a4-e106-4b1b-bf4a-f238bfea4d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:21.824894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253f7a4-e106-4b1b-bf4a-f238bfea4d3e service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:21.835353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1333/2670] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:21.849557 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1338/2671] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:21.857504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5794c7-f162-4f11-ba8c-d16450e603f8 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.857796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5794c7-f162-4f11-ba8c-d16450e603f8 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.858103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5794c7-f162-4f11-ba8c-d16450e603f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.877158 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-9a5794c7-f162-4f11-ba8c-d16450e603f8 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:21.878535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-9a5794c7-f162-4f11-ba8c-d16450e603f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:21.887084 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1334/2672] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:21.891657 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea91ab5-553c-4f64-8863-0fefa64551ed None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:21.897096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:21.903365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea91ab5-553c-4f64-8863-0fefa64551ed None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:21.911000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:21.911944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.912329 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.912683 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.913670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:06:21.914296 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:21.916167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:21.932538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1339/2673] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:06:21.933928 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5794c7-f162-4f11-ba8c-d16450e603f8 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:21.944624 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5794c7-f162-4f11-ba8c-d16450e603f8 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:21.950259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe4f8-fd87-4206-ba75-f3e276eb1668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:21.950854 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe4f8-fd87-4206-ba75-f3e276eb1668 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:21.951139 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe4f8-fd87-4206-ba75-f3e276eb1668 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:21.951423 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe4f8-fd87-4206-ba75-f3e276eb1668 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:21.952117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafe4f8-fd87-4206-ba75-f3e276eb1668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:21.954100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafe4f8-fd87-4206-ba75-f3e276eb1668 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:21.998172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1335/2674] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:21 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:22.001197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.012335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a3d9c-3b0a-4909-876a-1e4c28b7b845 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.074493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.075188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.075473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.075731 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.076429 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.078465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.106146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1340/2675] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:21 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:22.107189 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe4f8-fd87-4206-ba75-f3e276eb1668 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.118516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe4f8-fd87-4206-ba75-f3e276eb1668 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.122717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.124542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:22.125669 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75067b-3253-4f5e-a6fe-a334bf805fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.126674 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75067b-3253-4f5e-a6fe-a334bf805fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.127074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75067b-3253-4f5e-a6fe-a334bf805fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.127497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75067b-3253-4f5e-a6fe-a334bf805fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.128224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75067b-3253-4f5e-a6fe-a334bf805fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.130065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75067b-3253-4f5e-a6fe-a334bf805fd3 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.137905 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a16ad-9db8-4d29-bb17-80cc01584cc4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.143073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.143962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.144247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.144524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] PATH_INFO: `/v3/projects/24a8e10ef5c94505af867ddf2edefe56` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.145728 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a8e10ef5c94505af867ddf2edefe56)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.147726 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.154119 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1336/2676] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:22.158408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75067b-3253-4f5e-a6fe-a334bf805fd3 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.168565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75067b-3253-4f5e-a6fe-a334bf805fd3 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.174109 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36069d7-5f83-4ec5-b244-989e0c2b1a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.174783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36069d7-5f83-4ec5-b244-989e0c2b1a77 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.175066 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36069d7-5f83-4ec5-b244-989e0c2b1a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.175334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36069d7-5f83-4ec5-b244-989e0c2b1a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.176426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36069d7-5f83-4ec5-b244-989e0c2b1a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.178526 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36069d7-5f83-4ec5-b244-989e0c2b1a77 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.190130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1337/2677] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:22.194348 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36069d7-5f83-4ec5-b244-989e0c2b1a77 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.199998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a8e10ef5c94505af867ddf2edefe56', 'request_id': 'req-501d5bcc-8755-44d4-a96a-7551b8798bc7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:22.204527 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36069d7-5f83-4ec5-b244-989e0c2b1a77 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.209212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a8e10ef5c94505af867ddf2edefe56', 'request_id': 'req-501d5bcc-8755-44d4-a96a-7551b8798bc7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:22.216344 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a8e10ef5c94505af867ddf2edefe56', 'request_id': 'req-501d5bcc-8755-44d4-a96a-7551b8798bc7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:22.228111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] The token cache is being invalidate because project 24a8e10ef5c94505af867ddf2edefe56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:22.228373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:22.229074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-501d5bcc-8755-44d4-a96a-7551b8798bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:22.231133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1341/2678] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:22 2022] DELETE /identity/v3/projects/24a8e10ef5c94505af867ddf2edefe56 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:22.239650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bfec-bae4-4fe2-a019-b27256a6a26b None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.239928 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bfec-bae4-4fe2-a019-b27256a6a26b None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.240195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62bfec-bae4-4fe2-a019-b27256a6a26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.264993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:22.265295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-6e62bfec-bae4-4fe2-a019-b27256a6a26b None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:22.267298 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6e62bfec-bae4-4fe2-a019-b27256a6a26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:22.271882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759e8f-9ff0-44dc-9555-04d9961185d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.272789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759e8f-9ff0-44dc-9555-04d9961185d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.273080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759e8f-9ff0-44dc-9555-04d9961185d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.273530 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759e8f-9ff0-44dc-9555-04d9961185d1 admin admin] PATH_INFO: `/v3/projects/625cedb132b747a180213cfffcb5c1d6/users/bf12fe9624b54465bf39df2b810ce0df/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.276282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1342/2679] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:22.284556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1767-aac3-456c-8ce9-c80bfd3cb2f3 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.284844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1767-aac3-456c-8ce9-c80bfd3cb2f3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.285123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1767-aac3-456c-8ce9-c80bfd3cb2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.291147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94759e8f-9ff0-44dc-9555-04d9961185d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625cedb132b747a180213cfffcb5c1d6, user_id=bf12fe9624b54465bf39df2b810ce0df, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.294015 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94759e8f-9ff0-44dc-9555-04d9961185d1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.303481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3e6f1767-aac3-456c-8ce9-c80bfd3cb2f3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:22.304689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3e6f1767-aac3-456c-8ce9-c80bfd3cb2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:22.309333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1338/2680] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:22 2022] PUT /identity/v3/projects/625cedb132b747a180213cfffcb5c1d6/users/bf12fe9624b54465bf39df2b810ce0df/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:22.320615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999794f-cdcf-415c-8efb-1ac6f2e4cbec None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.320915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999794f-cdcf-415c-8efb-1ac6f2e4cbec None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.321193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999794f-cdcf-415c-8efb-1ac6f2e4cbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.351282 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-2999794f-cdcf-415c-8efb-1ac6f2e4cbec None None] MFA Rules not processed for user `bf12fe9624b54465bf39df2b810ce0df`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:22.352550 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2999794f-cdcf-415c-8efb-1ac6f2e4cbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:22.356763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:22.372665 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1343/2681] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:06:22.377380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f1767-aac3-456c-8ce9-c80bfd3cb2f3 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.386636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f1767-aac3-456c-8ce9-c80bfd3cb2f3 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.391624 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.392294 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.392601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.392877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.393613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:06:22.393956 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.395581 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.407322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1344/2682] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:06:22.411763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.422438 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cda65-5f29-40aa-ab27-6a6726fa0083 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.424263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1339/2683] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:06:22.427387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae491b0-7880-4b71-894f-d6b785fbd577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.428148 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae491b0-7880-4b71-894f-d6b785fbd577 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.428428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae491b0-7880-4b71-894f-d6b785fbd577 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.428632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999794f-cdcf-415c-8efb-1ac6f2e4cbec None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.428739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae491b0-7880-4b71-894f-d6b785fbd577 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.429556 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae491b0-7880-4b71-894f-d6b785fbd577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.429962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2999794f-cdcf-415c-8efb-1ac6f2e4cbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:22.431551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae491b0-7880-4b71-894f-d6b785fbd577 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.443373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999794f-cdcf-415c-8efb-1ac6f2e4cbec None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.448680 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495106fd-cb4e-4ad1-865e-f38e6da28140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.449473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495106fd-cb4e-4ad1-865e-f38e6da28140 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.449812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495106fd-cb4e-4ad1-865e-f38e6da28140 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.450186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495106fd-cb4e-4ad1-865e-f38e6da28140 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.450952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495106fd-cb4e-4ad1-865e-f38e6da28140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.453415 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495106fd-cb4e-4ad1-865e-f38e6da28140 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.459787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1345/2684] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:22.464596 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae491b0-7880-4b71-894f-d6b785fbd577 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.482562 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1340/2685] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:06:22.487444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495106fd-cb4e-4ad1-865e-f38e6da28140 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.557045 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae491b0-7880-4b71-894f-d6b785fbd577 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.581137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495106fd-cb4e-4ad1-865e-f38e6da28140 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.586984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32d60-834c-4cea-9719-69afc0db8161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.587679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32d60-834c-4cea-9719-69afc0db8161 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.588070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32d60-834c-4cea-9719-69afc0db8161 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.588232 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32d60-834c-4cea-9719-69afc0db8161 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.588926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f32d60-834c-4cea-9719-69afc0db8161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.591022 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f32d60-834c-4cea-9719-69afc0db8161 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.613787 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1341/2686] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:06:22.617620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32d60-834c-4cea-9719-69afc0db8161 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.624015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708d255-5df1-420a-ba0d-3c2f126abb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.624706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d255-5df1-420a-ba0d-3c2f126abb93 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.624993 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d255-5df1-420a-ba0d-3c2f126abb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.625520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d255-5df1-420a-ba0d-3c2f126abb93 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.626280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708d255-5df1-420a-ba0d-3c2f126abb93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.628565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708d255-5df1-420a-ba0d-3c2f126abb93 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.628706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32d60-834c-4cea-9719-69afc0db8161 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.633875 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3aed2-8438-48a2-8c96-5df42afb470a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.634542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3aed2-8438-48a2-8c96-5df42afb470a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.634824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3aed2-8438-48a2-8c96-5df42afb470a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.635100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3aed2-8438-48a2-8c96-5df42afb470a admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.635954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3aed2-8438-48a2-8c96-5df42afb470a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.637552 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3aed2-8438-48a2-8c96-5df42afb470a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.648648 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1342/2687] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:22.651328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1346/2688] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:22.652833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3aed2-8438-48a2-8c96-5df42afb470a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.655280 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708d255-5df1-420a-ba0d-3c2f126abb93 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.663793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3aed2-8438-48a2-8c96-5df42afb470a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.665873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708d255-5df1-420a-ba0d-3c2f126abb93 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.668724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12961cfb-f265-43e5-b10b-19ee99aab6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.669622 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12961cfb-f265-43e5-b10b-19ee99aab6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.669947 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12961cfb-f265-43e5-b10b-19ee99aab6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.670233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12961cfb-f265-43e5-b10b-19ee99aab6fe admin admin] PATH_INFO: `/v3/projects/aa81e8477a3a45f7a27ffc308980456d/users/8a6d4325e3c14bfdabc4c1940d54bf6f/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.671561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226ce8b-a3a6-4427-bc1f-f27c0d1aa2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.673082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226ce8b-a3a6-4427-bc1f-f27c0d1aa2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.673378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226ce8b-a3a6-4427-bc1f-f27c0d1aa2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.673672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226ce8b-a3a6-4427-bc1f-f27c0d1aa2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.674501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226ce8b-a3a6-4427-bc1f-f27c0d1aa2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.676185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226ce8b-a3a6-4427-bc1f-f27c0d1aa2f0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.686621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12961cfb-f265-43e5-b10b-19ee99aab6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa81e8477a3a45f7a27ffc308980456d, user_id=8a6d4325e3c14bfdabc4c1940d54bf6f, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.691180 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1347/2689] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:06:22.691313 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12961cfb-f265-43e5-b10b-19ee99aab6fe admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.692044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226ce8b-a3a6-4427-bc1f-f27c0d1aa2f0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.702543 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226ce8b-a3a6-4427-bc1f-f27c0d1aa2f0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.704542 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1343/2690] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:22 2022] PUT /identity/v3/projects/aa81e8477a3a45f7a27ffc308980456d/users/8a6d4325e3c14bfdabc4c1940d54bf6f/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:22.708373 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12961cfb-f265-43e5-b10b-19ee99aab6fe admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.712850 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:22.753174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:22.768237 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cc555-a65b-4a25-a40d-2214a4c52cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.768916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cc555-a65b-4a25-a40d-2214a4c52cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.769195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cc555-a65b-4a25-a40d-2214a4c52cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.769469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cc555-a65b-4a25-a40d-2214a4c52cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.770263 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cc555-a65b-4a25-a40d-2214a4c52cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.771916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cc555-a65b-4a25-a40d-2214a4c52cd8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.774924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12961cfb-f265-43e5-b10b-19ee99aab6fe admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.779952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb8443-f30b-446b-ba33-584bdeacb07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.780676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb8443-f30b-446b-ba33-584bdeacb07a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.780960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb8443-f30b-446b-ba33-584bdeacb07a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.781237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb8443-f30b-446b-ba33-584bdeacb07a admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.782173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb8443-f30b-446b-ba33-584bdeacb07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.782755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1348/2691] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:22.783735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb8443-f30b-446b-ba33-584bdeacb07a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.786853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cc555-a65b-4a25-a40d-2214a4c52cd8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.797818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1344/2692] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:22.798053 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cc555-a65b-4a25-a40d-2214a4c52cd8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.799358 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb8443-f30b-446b-ba33-584bdeacb07a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.802501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f36edd-8320-464a-b253-949b55deabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.803391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f36edd-8320-464a-b253-949b55deabdb admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.803674 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f36edd-8320-464a-b253-949b55deabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.803948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f36edd-8320-464a-b253-949b55deabdb admin admin] PATH_INFO: `/v3/projects/9e2cea3c89634da48a2178fd3e66cd4a/users/b51308e0724e40718d6bd9b909a52369/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.810294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb8443-f30b-446b-ba33-584bdeacb07a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:22.816195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c8695-438f-4c88-acd9-e998be672143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:22.817351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8695-438f-4c88-acd9-e998be672143 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.817351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8695-438f-4c88-acd9-e998be672143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.817791 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c8695-438f-4c88-acd9-e998be672143 admin admin] PATH_INFO: `/v3/projects/aa81e8477a3a45f7a27ffc308980456d/users/8a6d4325e3c14bfdabc4c1940d54bf6f/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.819671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f36edd-8320-464a-b253-949b55deabdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2cea3c89634da48a2178fd3e66cd4a, user_id=b51308e0724e40718d6bd9b909a52369, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.820352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c8695-438f-4c88-acd9-e998be672143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa81e8477a3a45f7a27ffc308980456d, user_id=8a6d4325e3c14bfdabc4c1940d54bf6f, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:22.822520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f36edd-8320-464a-b253-949b55deabdb admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.823136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c8695-438f-4c88-acd9-e998be672143 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:22.835793 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1345/2693] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:22 2022] PUT /identity/v3/projects/aa81e8477a3a45f7a27ffc308980456d/users/8a6d4325e3c14bfdabc4c1940d54bf6f/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:22.839214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1349/2694] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:22 2022] PUT /identity/v3/projects/9e2cea3c89634da48a2178fd3e66cd4a/users/b51308e0724e40718d6bd9b909a52369/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:22.846688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b364-65a3-4b3c-9a6b-ba61f68574fd None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.846980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b364-65a3-4b3c-9a6b-ba61f68574fd None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.847290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b364-65a3-4b3c-9a6b-ba61f68574fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.851043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64eece-9aaa-45a7-954e-c645e2725aaf None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:22.851332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64eece-9aaa-45a7-954e-c645e2725aaf None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:22.851619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64eece-9aaa-45a7-954e-c645e2725aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:22.877346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-ea18b364-65a3-4b3c-9a6b-ba61f68574fd None None] MFA Rules not processed for user `8a6d4325e3c14bfdabc4c1940d54bf6f`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:22.878631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ea18b364-65a3-4b3c-9a6b-ba61f68574fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:22.880895 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-ce64eece-9aaa-45a7-954e-c645e2725aaf None None] MFA Rules not processed for user `b51308e0724e40718d6bd9b909a52369`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:22.882099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ce64eece-9aaa-45a7-954e-c645e2725aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:22.955526 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1350/2695] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:06:22.955751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1346/2696] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:22.964498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64eece-9aaa-45a7-954e-c645e2725aaf None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.964842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18b364-65a3-4b3c-9a6b-ba61f68574fd None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:22.966071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ce64eece-9aaa-45a7-954e-c645e2725aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:22.966655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ea18b364-65a3-4b3c-9a6b-ba61f68574fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:23.035863 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18b364-65a3-4b3c-9a6b-ba61f68574fd None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:23.037510 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64eece-9aaa-45a7-954e-c645e2725aaf None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:23.039474 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97ba3-f0ed-424e-8f32-634ce6a78015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:23.040116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97ba3-f0ed-424e-8f32-634ce6a78015 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:23.040396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97ba3-f0ed-424e-8f32-634ce6a78015 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:23.040677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97ba3-f0ed-424e-8f32-634ce6a78015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:23.041331 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:23.042030 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:23.042313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:23.042715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:23.049703 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97ba3-f0ed-424e-8f32-634ce6a78015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:23.051177 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97ba3-f0ed-424e-8f32-634ce6a78015 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:23.052349 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:23.053856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:23.060082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1347/2697] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:06:23.063746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1351/2698] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:23.160408 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97ba3-f0ed-424e-8f32-634ce6a78015 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:23.170470 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97ba3-f0ed-424e-8f32-634ce6a78015 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:23.174377 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7c54f-8c91-416c-bbe0-bd1e3d6d4464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:23.175030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7c54f-8c91-416c-bbe0-bd1e3d6d4464 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:23.175351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7c54f-8c91-416c-bbe0-bd1e3d6d4464 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:23.175602 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7c54f-8c91-416c-bbe0-bd1e3d6d4464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:23.185388 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:23.246432 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7c54f-8c91-416c-bbe0-bd1e3d6d4464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:23.247899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7c54f-8c91-416c-bbe0-bd1e3d6d4464 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:23.257673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1348/2699] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:23 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:31.551983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:31.554256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:31.613223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:31.638883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028b2e4-1a30-495c-8fef-553070335d69 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:31.644204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:31.645245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:31.645704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:31.646768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] PATH_INFO: `/v3/users/3523cfc632b84b068751ca1a245de51e` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:31.648002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3523cfc632b84b068751ca1a245de51e)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:31.650168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:31.711223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3523cfc632b84b068751ca1a245de51e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:31.726374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3523cfc632b84b068751ca1a245de51e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:31.735573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3523cfc632b84b068751ca1a245de51e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:31.736509 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3523cfc632b84b068751ca1a245de51e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:31.746305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3523cfc632b84b068751ca1a245de51e'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:31.753154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1352/2700] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:31 2022] DELETE /identity/v3/users/3523cfc632b84b068751ca1a245de51e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:31.759785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29cea-9d81-48b9-9f1a-0bb8a89b6c71 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:31.764459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29cea-9d81-48b9-9f1a-0bb8a89b6c71 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:31.764459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c29cea-9d81-48b9-9f1a-0bb8a89b6c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:31.783341 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-a0c29cea-9d81-48b9-9f1a-0bb8a89b6c71 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:31.784586 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a0c29cea-9d81-48b9-9f1a-0bb8a89b6c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:31.872767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1349/2701] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:31.885358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:31.887058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:31.980197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc2bd4-47c6-418c-a825-bfefe6703801 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:31.987152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da569a-f64a-42e4-a668-24f9c6647ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:31.988460 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da569a-f64a-42e4-a668-24f9c6647ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:31.988987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da569a-f64a-42e4-a668-24f9c6647ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:31.989579 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da569a-f64a-42e4-a668-24f9c6647ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:32.005693 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da569a-f64a-42e4-a668-24f9c6647ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:32.007396 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da569a-f64a-42e4-a668-24f9c6647ac4 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:32.021839 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1353/2702] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:32.407126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c29cea-9d81-48b9-9f1a-0bb8a89b6c71 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:32.408984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a0c29cea-9d81-48b9-9f1a-0bb8a89b6c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:32.422513 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c29cea-9d81-48b9-9f1a-0bb8a89b6c71 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:32.428220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:32.429122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:32.429404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:32.429684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] PATH_INFO: `/v3/projects/e608adfcfb4c4786ac1220411b0fa10d` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:32.430993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e608adfcfb4c4786ac1220411b0fa10d)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:32.432946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:32.497545 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e608adfcfb4c4786ac1220411b0fa10d', 'request_id': 'req-42e8e729-4b2d-4adc-bdfe-66a174159e52'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.506319 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e608adfcfb4c4786ac1220411b0fa10d', 'request_id': 'req-42e8e729-4b2d-4adc-bdfe-66a174159e52'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.516758 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e608adfcfb4c4786ac1220411b0fa10d', 'request_id': 'req-42e8e729-4b2d-4adc-bdfe-66a174159e52'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.534757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] The token cache is being invalidate because project e608adfcfb4c4786ac1220411b0fa10d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:32.535058 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.535740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.537426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1350/2703] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:32 2022] DELETE /identity/v3/projects/e608adfcfb4c4786ac1220411b0fa10d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:32.542175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da569a-f64a-42e4-a668-24f9c6647ac4 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:32.553812 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da569a-f64a-42e4-a668-24f9c6647ac4 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:32.622745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:32.623777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:32.624128 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:32.624477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] PATH_INFO: `/v3/users/02cd229cba2e49a4807d07f2876c6be7` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:32.625810 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02cd229cba2e49a4807d07f2876c6be7)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:32.627970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:32.647675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:32.693588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02cd229cba2e49a4807d07f2876c6be7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.705410 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02cd229cba2e49a4807d07f2876c6be7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.714824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02cd229cba2e49a4807d07f2876c6be7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.715493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02cd229cba2e49a4807d07f2876c6be7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.722872 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2612e63e-781c-4fe6-b5e6-d157d7a1d2e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02cd229cba2e49a4807d07f2876c6be7'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:32.730142 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1354/2704] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:32 2022] DELETE /identity/v3/users/02cd229cba2e49a4807d07f2876c6be7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:33.044206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:33.047275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:33.074911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8e729-4b2d-4adc-bdfe-66a174159e52 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:33.137277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:33.172303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:33.173635 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.174113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.174446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] PATH_INFO: `/v3/projects/ae2fa1a6691f4a069c375062eb7e6a84` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.176241 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae2fa1a6691f4a069c375062eb7e6a84)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:33.179187 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:33.244904 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae2fa1a6691f4a069c375062eb7e6a84', 'request_id': 'req-a54ed566-2ea8-43f2-87e0-6da1e00ee823'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:33.258417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae2fa1a6691f4a069c375062eb7e6a84', 'request_id': 'req-a54ed566-2ea8-43f2-87e0-6da1e00ee823'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:33.270838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae2fa1a6691f4a069c375062eb7e6a84', 'request_id': 'req-a54ed566-2ea8-43f2-87e0-6da1e00ee823'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:33.283809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] The token cache is being invalidate because project ae2fa1a6691f4a069c375062eb7e6a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:33.284091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:33.284806 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a54ed566-2ea8-43f2-87e0-6da1e00ee823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:33.286661 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1351/2705] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:33 2022] DELETE /identity/v3/projects/ae2fa1a6691f4a069c375062eb7e6a84 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:33.292827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d91af9-4d2c-4df7-b068-521d393a3c76 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.293113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d91af9-4d2c-4df7-b068-521d393a3c76 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.293392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d91af9-4d2c-4df7-b068-521d393a3c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.316245 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e8d91af9-4d2c-4df7-b068-521d393a3c76 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:33.317541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e8d91af9-4d2c-4df7-b068-521d393a3c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:33.394546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1355/2706] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:33.404696 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558cfa-f248-40c0-8623-854f2c238599 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.405033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558cfa-f248-40c0-8623-854f2c238599 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.405275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f558cfa-f248-40c0-8623-854f2c238599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.427033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-6f558cfa-f248-40c0-8623-854f2c238599 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:33.428265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6f558cfa-f248-40c0-8623-854f2c238599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:33.437563 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1352/2707] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:33.442386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d91af9-4d2c-4df7-b068-521d393a3c76 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:33.454125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d91af9-4d2c-4df7-b068-521d393a3c76 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:33.460151 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:33.460869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.461168 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.461462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.462517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:06:33.462967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:33.465719 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:33.484109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1356/2708] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:06:33.490186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f558cfa-f248-40c0-8623-854f2c238599 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:33.504842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f558cfa-f248-40c0-8623-854f2c238599 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:33.511234 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfca9e-d032-4b75-9dd3-1d5ec34bdda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:33.511971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfca9e-d032-4b75-9dd3-1d5ec34bdda5 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.512247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfca9e-d032-4b75-9dd3-1d5ec34bdda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.512517 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfca9e-d032-4b75-9dd3-1d5ec34bdda5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.513263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfca9e-d032-4b75-9dd3-1d5ec34bdda5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:33.515368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfca9e-d032-4b75-9dd3-1d5ec34bdda5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:33.551010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1353/2709] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:33 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:33.556618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:33.573204 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb039-5ca3-4234-a8e7-e90efcfdc888 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:33.652983 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f4c87-77b1-4c04-96a0-38c0673599b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:33.653678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f4c87-77b1-4c04-96a0-38c0673599b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.654462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f4c87-77b1-4c04-96a0-38c0673599b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.654462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f4c87-77b1-4c04-96a0-38c0673599b6 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.655125 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f4c87-77b1-4c04-96a0-38c0673599b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:33.657177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f4c87-77b1-4c04-96a0-38c0673599b6 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:33.684197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1357/2710] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:33 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:33.689689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfca9e-d032-4b75-9dd3-1d5ec34bdda5 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:33.707848 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfca9e-d032-4b75-9dd3-1d5ec34bdda5 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:33.714429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cefb2-0c59-410d-91ca-f402897ee71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:33.715120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cefb2-0c59-410d-91ca-f402897ee71a admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.715398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cefb2-0c59-410d-91ca-f402897ee71a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.715668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cefb2-0c59-410d-91ca-f402897ee71a admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.716456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cefb2-0c59-410d-91ca-f402897ee71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:33.718140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cefb2-0c59-410d-91ca-f402897ee71a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:33.731296 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1354/2711] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:33.737228 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f4c87-77b1-4c04-96a0-38c0673599b6 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:33.749663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f4c87-77b1-4c04-96a0-38c0673599b6 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:33.755715 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0364cb2-a552-4064-9c2a-e4c878517966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:33.756603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364cb2-a552-4064-9c2a-e4c878517966 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.756813 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364cb2-a552-4064-9c2a-e4c878517966 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.756979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364cb2-a552-4064-9c2a-e4c878517966 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.757756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0364cb2-a552-4064-9c2a-e4c878517966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:33.759653 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0364cb2-a552-4064-9c2a-e4c878517966 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:33.763050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:33.776034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1358/2712] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:33.781034 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cefb2-0c59-410d-91ca-f402897ee71a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:33.793458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cefb2-0c59-410d-91ca-f402897ee71a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:33.802792 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7fedb-764a-499a-9bfb-89dd232e7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:33.803810 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7fedb-764a-499a-9bfb-89dd232e7527 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.804082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7fedb-764a-499a-9bfb-89dd232e7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.804344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7fedb-764a-499a-9bfb-89dd232e7527 admin admin] PATH_INFO: `/v3/projects/e7af49ecea2c419c8afb779174fe5d47/users/3f295ccb6277463882b2c10f5cd8674a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.823228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7fedb-764a-499a-9bfb-89dd232e7527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7af49ecea2c419c8afb779174fe5d47, user_id=3f295ccb6277463882b2c10f5cd8674a, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:33.827910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7fedb-764a-499a-9bfb-89dd232e7527 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:33.846546 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1355/2713] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:33 2022] PUT /identity/v3/projects/e7af49ecea2c419c8afb779174fe5d47/users/3f295ccb6277463882b2c10f5cd8674a/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:33.860039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f58c3-cc96-4aff-bff5-e42f0be228af None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:33.860370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f58c3-cc96-4aff-bff5-e42f0be228af None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:33.860649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f58c3-cc96-4aff-bff5-e42f0be228af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:33.892710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-696f58c3-cc96-4aff-bff5-e42f0be228af None None] MFA Rules not processed for user `3f295ccb6277463882b2c10f5cd8674a`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:33.894019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-696f58c3-cc96-4aff-bff5-e42f0be228af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:33.975077 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1359/2714] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:33.980075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7fedb-764a-499a-9bfb-89dd232e7527 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:34.062493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7fedb-764a-499a-9bfb-89dd232e7527 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:34.068100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117916c8-1608-4445-9984-86f6d5942df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:34.069347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117916c8-1608-4445-9984-86f6d5942df0 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:34.069652 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117916c8-1608-4445-9984-86f6d5942df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:34.070100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117916c8-1608-4445-9984-86f6d5942df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:34.070872 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117916c8-1608-4445-9984-86f6d5942df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:34.072898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117916c8-1608-4445-9984-86f6d5942df0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:34.078173 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:34.116677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1356/2715] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:33 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:34.121518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f58c3-cc96-4aff-bff5-e42f0be228af None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:34.134970 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f58c3-cc96-4aff-bff5-e42f0be228af None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:34.206378 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c63237-5520-4bdc-8c76-3cf97e6d16c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:34.207104 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63237-5520-4bdc-8c76-3cf97e6d16c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:34.207371 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63237-5520-4bdc-8c76-3cf97e6d16c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:34.207651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c63237-5520-4bdc-8c76-3cf97e6d16c2 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:34.208493 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c63237-5520-4bdc-8c76-3cf97e6d16c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:34.210744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c63237-5520-4bdc-8c76-3cf97e6d16c2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:34.236281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1360/2716] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:34 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:34.240887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117916c8-1608-4445-9984-86f6d5942df0 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:34.254673 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117916c8-1608-4445-9984-86f6d5942df0 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:34.259783 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b4687-87d2-41f9-9f39-d848b445460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:34.260346 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b4687-87d2-41f9-9f39-d848b445460d admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:34.260627 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b4687-87d2-41f9-9f39-d848b445460d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:34.260927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b4687-87d2-41f9-9f39-d848b445460d admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:34.261679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b4687-87d2-41f9-9f39-d848b445460d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:34.263420 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b4687-87d2-41f9-9f39-d848b445460d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:34.276837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1357/2717] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:34.282020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c63237-5520-4bdc-8c76-3cf97e6d16c2 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:34.293669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c63237-5520-4bdc-8c76-3cf97e6d16c2 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:34.298844 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a494-2f31-4003-8298-0a5450b12cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:34.299732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a494-2f31-4003-8298-0a5450b12cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:34.300010 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a494-2f31-4003-8298-0a5450b12cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:34.300279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a494-2f31-4003-8298-0a5450b12cbe admin admin] PATH_INFO: `/v3/projects/70637dfc9f1a4aa5a2a01215edfa261b/users/4cc76faba04c41f58bb0e3c194e844f0/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:34.316538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54a494-2f31-4003-8298-0a5450b12cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=70637dfc9f1a4aa5a2a01215edfa261b, user_id=4cc76faba04c41f58bb0e3c194e844f0, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:34.319910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54a494-2f31-4003-8298-0a5450b12cbe admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:34.334745 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1361/2718] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:34 2022] PUT /identity/v3/projects/70637dfc9f1a4aa5a2a01215edfa261b/users/4cc76faba04c41f58bb0e3c194e844f0/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:34.338992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b4687-87d2-41f9-9f39-d848b445460d admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:34.351550 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b4687-87d2-41f9-9f39-d848b445460d admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:34.420304 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a12c3-4be6-4e21-8847-d9919b6dbfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:34.421013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a12c3-4be6-4e21-8847-d9919b6dbfb6 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:34.421301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a12c3-4be6-4e21-8847-d9919b6dbfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:34.421577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a12c3-4be6-4e21-8847-d9919b6dbfb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:34.422402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a12c3-4be6-4e21-8847-d9919b6dbfb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:34.424097 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a12c3-4be6-4e21-8847-d9919b6dbfb6 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:34.436174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1358/2719] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:34.441024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a494-2f31-4003-8298-0a5450b12cbe admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:34.452405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a494-2f31-4003-8298-0a5450b12cbe admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:34.461066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:34.462056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:34.462335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:34.462608 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] PATH_INFO: `/v3/projects/70637dfc9f1a4aa5a2a01215edfa261b/users/4cc76faba04c41f58bb0e3c194e844f0/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:34.464914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70637dfc9f1a4aa5a2a01215edfa261b, user_id=4cc76faba04c41f58bb0e3c194e844f0, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:34.467649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:34.482089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1362/2720] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:34 2022] PUT /identity/v3/projects/70637dfc9f1a4aa5a2a01215edfa261b/users/4cc76faba04c41f58bb0e3c194e844f0/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:34.493028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f3eb-5102-48f0-9ab6-fd5bd371cdfc None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:34.493278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f3eb-5102-48f0-9ab6-fd5bd371cdfc None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:34.493844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f3eb-5102-48f0-9ab6-fd5bd371cdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:34.527881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-6c28f3eb-5102-48f0-9ab6-fd5bd371cdfc None None] MFA Rules not processed for user `4cc76faba04c41f58bb0e3c194e844f0`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:34.529069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-6c28f3eb-5102-48f0-9ab6-fd5bd371cdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:34.620651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1359/2721] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:34.635584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:34.637957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:34.704824 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:34.740066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05ad83-0a2c-4582-b6c0-a8c37e839bf3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:34.744293 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d652aa-fa1d-4004-a6fa-77af3055d232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:34.744962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d652aa-fa1d-4004-a6fa-77af3055d232 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:34.745235 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d652aa-fa1d-4004-a6fa-77af3055d232 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:34.745505 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d652aa-fa1d-4004-a6fa-77af3055d232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:34.755102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d652aa-fa1d-4004-a6fa-77af3055d232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:34.756855 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d652aa-fa1d-4004-a6fa-77af3055d232 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:34.766679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1363/2722] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:35.375165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f3eb-5102-48f0-9ab6-fd5bd371cdfc None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:35.449749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:35.454588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f3eb-5102-48f0-9ab6-fd5bd371cdfc None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:35.460646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:35.461859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:35.462154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:35.462433 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] PATH_INFO: `/v3/users/3f295ccb6277463882b2c10f5cd8674a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:35.463690 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f295ccb6277463882b2c10f5cd8674a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:35.465983 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:35.521607 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f295ccb6277463882b2c10f5cd8674a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:35.536847 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f295ccb6277463882b2c10f5cd8674a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:35.548504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f295ccb6277463882b2c10f5cd8674a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:35.556899 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f295ccb6277463882b2c10f5cd8674a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:35.557614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f295ccb6277463882b2c10f5cd8674a'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:35.564427 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1360/2723] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:35 2022] DELETE /identity/v3/users/3f295ccb6277463882b2c10f5cd8674a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:35.569712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0766663-3ef1-43cb-91f6-a6475c6c1ee7 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:35.570027 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0766663-3ef1-43cb-91f6-a6475c6c1ee7 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:35.570301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0766663-3ef1-43cb-91f6-a6475c6c1ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:35.588772 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b0766663-3ef1-43cb-91f6-a6475c6c1ee7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:35.590062 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b0766663-3ef1-43cb-91f6-a6475c6c1ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:35.668492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1364/2724] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:35.681144 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:35.683368 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:35.753497 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e49cd0-fe41-4507-9f2e-2a7647fde308 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:35.758561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fb276-651d-47d1-83a9-84ae00fd88ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:35.759266 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb276-651d-47d1-83a9-84ae00fd88ea service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:35.759553 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb276-651d-47d1-83a9-84ae00fd88ea service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:35.759822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fb276-651d-47d1-83a9-84ae00fd88ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:35.771340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fb276-651d-47d1-83a9-84ae00fd88ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:35.772936 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fb276-651d-47d1-83a9-84ae00fd88ea service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:35.782796 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1361/2725] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:36.071621 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0766663-3ef1-43cb-91f6-a6475c6c1ee7 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:36.073358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b0766663-3ef1-43cb-91f6-a6475c6c1ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:36.089646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0766663-3ef1-43cb-91f6-a6475c6c1ee7 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:36.094689 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:36.095602 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:36.095880 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:36.096166 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] PATH_INFO: `/v3/projects/e7af49ecea2c419c8afb779174fe5d47` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:36.097375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7af49ecea2c419c8afb779174fe5d47)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:36.099335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:36.153701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7af49ecea2c419c8afb779174fe5d47', 'request_id': 'req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.163278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7af49ecea2c419c8afb779174fe5d47', 'request_id': 'req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.170789 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7af49ecea2c419c8afb779174fe5d47', 'request_id': 'req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.181854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] The token cache is being invalidate because project e7af49ecea2c419c8afb779174fe5d47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:36.182145 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.182962 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.184542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1365/2726] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:36 2022] DELETE /identity/v3/projects/e7af49ecea2c419c8afb779174fe5d47 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:36.188845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fb276-651d-47d1-83a9-84ae00fd88ea service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:36.207414 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fb276-651d-47d1-83a9-84ae00fd88ea service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:36.279996 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:36.280892 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:36.281176 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:36.281454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] PATH_INFO: `/v3/users/4cc76faba04c41f58bb0e3c194e844f0` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:36.282805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc76faba04c41f58bb0e3c194e844f0)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:36.284754 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:36.309819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:36.347662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc76faba04c41f58bb0e3c194e844f0'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.364814 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc76faba04c41f58bb0e3c194e844f0'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.374138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc76faba04c41f58bb0e3c194e844f0'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.387245 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc76faba04c41f58bb0e3c194e844f0'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.387898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-c02bee29-fdf0-4c33-b5ca-205486fec241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc76faba04c41f58bb0e3c194e844f0'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.393942 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1362/2727] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:36 2022] DELETE /identity/v3/users/4cc76faba04c41f58bb0e3c194e844f0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:36.687177 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:36.688652 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:36.707704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb61b69-520d-42c9-ac2c-15a48b5b4f31 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:36.760694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:36.778667 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:36.779569 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:36.779850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:36.780164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] PATH_INFO: `/v3/projects/70637dfc9f1a4aa5a2a01215edfa261b` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:36.781418 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70637dfc9f1a4aa5a2a01215edfa261b)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:36.783328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:36.843725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70637dfc9f1a4aa5a2a01215edfa261b', 'request_id': 'req-c8d2eae9-5be0-4faf-849a-d7f3b4307754'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.854071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70637dfc9f1a4aa5a2a01215edfa261b', 'request_id': 'req-c8d2eae9-5be0-4faf-849a-d7f3b4307754'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.861089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70637dfc9f1a4aa5a2a01215edfa261b', 'request_id': 'req-c8d2eae9-5be0-4faf-849a-d7f3b4307754'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.873982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] The token cache is being invalidate because project 70637dfc9f1a4aa5a2a01215edfa261b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:36.874252 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.875155 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c8d2eae9-5be0-4faf-849a-d7f3b4307754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:36.876941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1366/2728] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:36 2022] DELETE /identity/v3/projects/70637dfc9f1a4aa5a2a01215edfa261b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:36.886397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b15a9-e02e-4f19-8d2e-69d03b7e2a7e None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:36.886888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b15a9-e02e-4f19-8d2e-69d03b7e2a7e None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:36.887307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b15a9-e02e-4f19-8d2e-69d03b7e2a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:36.909074 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f80b15a9-e02e-4f19-8d2e-69d03b7e2a7e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:36.910318 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f80b15a9-e02e-4f19-8d2e-69d03b7e2a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:36.995784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1363/2729] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:37.003934 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837343fd-38fd-4dcc-a5c7-0f35a17b3f9c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.004274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837343fd-38fd-4dcc-a5c7-0f35a17b3f9c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.004483 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837343fd-38fd-4dcc-a5c7-0f35a17b3f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.032670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-837343fd-38fd-4dcc-a5c7-0f35a17b3f9c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:37.034434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-837343fd-38fd-4dcc-a5c7-0f35a17b3f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:37.044734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1367/2730] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:37.049822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b15a9-e02e-4f19-8d2e-69d03b7e2a7e None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:37.065064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b15a9-e02e-4f19-8d2e-69d03b7e2a7e None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:37.072326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:37.073107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.073592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.073689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.074495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:06:37.074807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:37.081338 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:37.097475 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1364/2731] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:06:37.102750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837343fd-38fd-4dcc-a5c7-0f35a17b3f9c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:37.115981 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837343fd-38fd-4dcc-a5c7-0f35a17b3f9c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:37.121878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e24e-5a54-423c-8933-505da8f453b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:37.122549 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e24e-5a54-423c-8933-505da8f453b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.122828 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e24e-5a54-423c-8933-505da8f453b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.123103 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e24e-5a54-423c-8933-505da8f453b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.123801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e24e-5a54-423c-8933-505da8f453b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:37.125725 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e24e-5a54-423c-8933-505da8f453b2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:37.157118 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1368/2732] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:37 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:37.164734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:37.175782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a997c-cbc9-4a35-a75d-2d905af2d4ed admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:37.243327 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2b5b1-bdfc-423f-ba06-d78d1798b858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:37.244020 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2b5b1-bdfc-423f-ba06-d78d1798b858 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.244300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2b5b1-bdfc-423f-ba06-d78d1798b858 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.244574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2b5b1-bdfc-423f-ba06-d78d1798b858 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.245324 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2b5b1-bdfc-423f-ba06-d78d1798b858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:37.247926 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2b5b1-bdfc-423f-ba06-d78d1798b858 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:37.275274 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1365/2733] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:37 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:37.279328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e24e-5a54-423c-8933-505da8f453b2 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:37.294006 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e24e-5a54-423c-8933-505da8f453b2 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:37.296409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526b9c0-6f0b-4c9b-a3da-8f46bcce5574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:37.297271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b9c0-6f0b-4c9b-a3da-8f46bcce5574 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.297620 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b9c0-6f0b-4c9b-a3da-8f46bcce5574 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.297990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526b9c0-6f0b-4c9b-a3da-8f46bcce5574 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.298875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526b9c0-6f0b-4c9b-a3da-8f46bcce5574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:37.300562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526b9c0-6f0b-4c9b-a3da-8f46bcce5574 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:37.312639 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1369/2734] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:06:37.316843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2b5b1-bdfc-423f-ba06-d78d1798b858 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:37.327540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2b5b1-bdfc-423f-ba06-d78d1798b858 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:37.333646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866ae5e-60bb-4f56-875a-c38a703e08ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:37.334407 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ae5e-60bb-4f56-875a-c38a703e08ff admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.334693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ae5e-60bb-4f56-875a-c38a703e08ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.334975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866ae5e-60bb-4f56-875a-c38a703e08ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.335770 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866ae5e-60bb-4f56-875a-c38a703e08ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:37.337444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866ae5e-60bb-4f56-875a-c38a703e08ff admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:37.349105 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1366/2735] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:37.355832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526b9c0-6f0b-4c9b-a3da-8f46bcce5574 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:37.370222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526b9c0-6f0b-4c9b-a3da-8f46bcce5574 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:37.376110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:37.377098 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.377443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.377739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] PATH_INFO: `/v3/projects/90cde3b10b264b0b8fb49d15129f5070/users/50619a890c0c4f09ac047d44bd842451/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.397365 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90cde3b10b264b0b8fb49d15129f5070, user_id=50619a890c0c4f09ac047d44bd842451, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:37.400119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:37.416013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1370/2736] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:37 2022] PUT /identity/v3/projects/90cde3b10b264b0b8fb49d15129f5070/users/50619a890c0c4f09ac047d44bd842451/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:37.428303 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d738b-0c71-4738-b1d4-49f9a19e3ebe None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.429005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d738b-0c71-4738-b1d4-49f9a19e3ebe None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.429005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d738b-0c71-4738-b1d4-49f9a19e3ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.439617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:37.467100 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-920d738b-0c71-4738-b1d4-49f9a19e3ebe None None] MFA Rules not processed for user `50619a890c0c4f09ac047d44bd842451`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:37.468435 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-920d738b-0c71-4738-b1d4-49f9a19e3ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:37.551501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1367/2737] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:37.564364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:37.566498 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:37.630827 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88e239-57fa-4591-8d4c-86c123026ae8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:37.636386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:37.637332 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:37.637691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:37.638158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:37.652599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:37.655249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:37.667313 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1371/2738] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:47.548244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=30 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:47.661542 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:49.093207 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d738b-0c71-4738-b1d4-49f9a19e3ebe None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:49.095499 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-920d738b-0c71-4738-b1d4-49f9a19e3ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:49.207347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d738b-0c71-4738-b1d4-49f9a19e3ebe None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:49.213153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:49.213842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:49.214147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:49.214406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:49.216410 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:49.229540 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:49.231292 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:49.242745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1368/2739] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:52.053260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:52.055002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.143063 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9aa81-940a-47a0-b0d5-a3a479a777a9 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:52.148475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:52.149889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.150188 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.150455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] PATH_INFO: `/v3/users/bf12fe9624b54465bf39df2b810ce0df` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.151887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf12fe9624b54465bf39df2b810ce0df)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:52.154202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:52.180188 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:52.181684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.197488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6963cc2-5f56-4f2f-911f-b2a08e8f9bca service placement] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:52.212145 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:52.213059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.213344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.213619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] PATH_INFO: `/v3/users/b51308e0724e40718d6bd9b909a52369` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.215056 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51308e0724e40718d6bd9b909a52369)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:52.217003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:52.217702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf12fe9624b54465bf39df2b810ce0df'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.231284 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf12fe9624b54465bf39df2b810ce0df'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.241195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf12fe9624b54465bf39df2b810ce0df'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.241932 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf12fe9624b54465bf39df2b810ce0df'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.253366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf12fe9624b54465bf39df2b810ce0df'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.265462 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1372/2740] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:52 2022] DELETE /identity/v3/users/bf12fe9624b54465bf39df2b810ce0df => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:52.266554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:52.268017 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.285164 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51308e0724e40718d6bd9b909a52369'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.298817 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51308e0724e40718d6bd9b909a52369'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.307715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51308e0724e40718d6bd9b909a52369'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.317393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51308e0724e40718d6bd9b909a52369'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.318417 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-9ee261fb-e265-4b87-b4f9-e4a9831a5a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51308e0724e40718d6bd9b909a52369'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.325473 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1369/2741] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:52 2022] DELETE /identity/v3/users/b51308e0724e40718d6bd9b909a52369 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:52.327194 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873cac-e512-40e0-ac30-f38e6e1b6fac None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.327592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873cac-e512-40e0-ac30-f38e6e1b6fac None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.327864 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873cac-e512-40e0-ac30-f38e6e1b6fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.338387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad5c7-02b2-4646-a1f7-54859e03da7c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:52.346382 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-09873cac-e512-40e0-ac30-f38e6e1b6fac None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:52.347641 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-09873cac-e512-40e0-ac30-f38e6e1b6fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.365461 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:52.395068 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:52.414194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:52.415096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.415373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.415647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] PATH_INFO: `/v3/users/50619a890c0c4f09ac047d44bd842451` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.416866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50619a890c0c4f09ac047d44bd842451)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:52.419008 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:52.421724 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1370/2742] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:52.422880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f9ce4-7f66-4018-878a-a9213f141daa None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.423235 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f9ce4-7f66-4018-878a-a9213f141daa None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.423608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f9ce4-7f66-4018-878a-a9213f141daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.442626 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b96f9ce4-7f66-4018-878a-a9213f141daa None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:52.444107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b96f9ce4-7f66-4018-878a-a9213f141daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.452590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1371/2743] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:52.453712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f9ce4-7f66-4018-878a-a9213f141daa None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:52.455059 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b96f9ce4-7f66-4018-878a-a9213f141daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.479130 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50619a890c0c4f09ac047d44bd842451'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.492246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50619a890c0c4f09ac047d44bd842451'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.502892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50619a890c0c4f09ac047d44bd842451'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.503551 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50619a890c0c4f09ac047d44bd842451'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.512126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50619a890c0c4f09ac047d44bd842451'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:52.519870 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1373/2744] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:52 2022] DELETE /identity/v3/users/50619a890c0c4f09ac047d44bd842451 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:52.520763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:52.522227 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.522749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f9ce4-7f66-4018-878a-a9213f141daa None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:52.580212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc131c-bd26-4f16-aeca-a4d82bade04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:52.580971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc131c-bd26-4f16-aeca-a4d82bade04f service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.581372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc131c-bd26-4f16-aeca-a4d82bade04f service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.581738 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc131c-bd26-4f16-aeca-a4d82bade04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.583620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-05bc131c-bd26-4f16-aeca-a4d82bade04f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.593383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75598d-dad2-43bc-bb4b-39fc1432f4e0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:52.598137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:52.598829 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.599114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.599398 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.601648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.654941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc131c-bd26-4f16-aeca-a4d82bade04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:52.656349 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc131c-bd26-4f16-aeca-a4d82bade04f service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:52.666386 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1372/2745] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:52.667844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a82d-4088-4989-b354-d162e1b7de7f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.668142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a82d-4088-4989-b354-d162e1b7de7f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.668425 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416a82d-4088-4989-b354-d162e1b7de7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.677911 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:52.680739 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:52.682383 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:52.688026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8416a82d-4088-4989-b354-d162e1b7de7f None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:52.689668 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8416a82d-4088-4989-b354-d162e1b7de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:52.695931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1374/2746] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:52.700759 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1373/2747] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:52.712454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:52.723366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96126d55-a8a4-4d55-84be-30dfa1bbb8ec service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:52.728242 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:52.729025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:52.729468 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:52.729991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:52.743506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:52.744883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:52.756436 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1375/2748] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:53.208404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416a82d-4088-4989-b354-d162e1b7de7f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:53.209938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8416a82d-4088-4989-b354-d162e1b7de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:53.222819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416a82d-4088-4989-b354-d162e1b7de7f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:53.228380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:53.229169 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:53.229456 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:53.229734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] PATH_INFO: `/v3/projects/9e2cea3c89634da48a2178fd3e66cd4a` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:53.231037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e2cea3c89634da48a2178fd3e66cd4a)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:53.233200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:53.275752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:53.277470 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:53.300609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:53.310763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca60a9-0f45-4908-a25f-813bc1426393 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:53.321117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e2cea3c89634da48a2178fd3e66cd4a', 'request_id': 'req-3608765c-6887-4ecb-8eb4-716cf7f7130e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.329419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e2cea3c89634da48a2178fd3e66cd4a', 'request_id': 'req-3608765c-6887-4ecb-8eb4-716cf7f7130e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.340707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e2cea3c89634da48a2178fd3e66cd4a', 'request_id': 'req-3608765c-6887-4ecb-8eb4-716cf7f7130e'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.353625 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] The token cache is being invalidate because project 9e2cea3c89634da48a2178fd3e66cd4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:53.353994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.354629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.356567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1374/2749] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:53 2022] DELETE /identity/v3/projects/9e2cea3c89634da48a2178fd3e66cd4a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:53.357172 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:53.358992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:53.404964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:53.406185 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:53.406607 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:53.406980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] PATH_INFO: `/v3/projects/625cedb132b747a180213cfffcb5c1d6` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:53.409056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625cedb132b747a180213cfffcb5c1d6)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:53.411327 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:53.440224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608765c-6887-4ecb-8eb4-716cf7f7130e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:53.446812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:53.447837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:53.448171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:53.448444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] PATH_INFO: `/v3/projects/90cde3b10b264b0b8fb49d15129f5070` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:53.449922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90cde3b10b264b0b8fb49d15129f5070)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:53.452203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:53.481640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625cedb132b747a180213cfffcb5c1d6', 'request_id': 'req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.491538 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625cedb132b747a180213cfffcb5c1d6', 'request_id': 'req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.501195 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625cedb132b747a180213cfffcb5c1d6', 'request_id': 'req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.516721 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] The token cache is being invalidate because project 625cedb132b747a180213cfffcb5c1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:53.517106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.517737 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.522450 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1376/2750] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:53 2022] DELETE /identity/v3/projects/625cedb132b747a180213cfffcb5c1d6 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:53.523903 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:53.524054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90cde3b10b264b0b8fb49d15129f5070', 'request_id': 'req-7699f74d-addd-4d05-8076-81801ef1f91c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.525475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:53.533228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90cde3b10b264b0b8fb49d15129f5070', 'request_id': 'req-7699f74d-addd-4d05-8076-81801ef1f91c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.544147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90cde3b10b264b0b8fb49d15129f5070', 'request_id': 'req-7699f74d-addd-4d05-8076-81801ef1f91c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.562030 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] The token cache is being invalidate because project 90cde3b10b264b0b8fb49d15129f5070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:53.562565 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.563484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-7699f74d-addd-4d05-8076-81801ef1f91c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.565704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1375/2751] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:53 2022] DELETE /identity/v3/projects/90cde3b10b264b0b8fb49d15129f5070 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:53.570974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192093-33b0-4926-ae42-bb2f10441c84 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:53.571340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192093-33b0-4926-ae42-bb2f10441c84 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:53.571682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192093-33b0-4926-ae42-bb2f10441c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:53.599267 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-03192093-33b0-4926-ae42-bb2f10441c84 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:53.601629 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-03192093-33b0-4926-ae42-bb2f10441c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:53.615850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a6b03-1d2b-41c7-b78a-be7a78afdcd5 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:53.643488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:53.662111 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:53.697442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1376/2752] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:53.700256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0b17-f7ba-465e-bd1a-c3a7417a52d0 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:53.701422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0b17-f7ba-465e-bd1a-c3a7417a52d0 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:53.701799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d0b17-f7ba-465e-bd1a-c3a7417a52d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:53.711102 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:53.712469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:53.712921 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:53.713256 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] PATH_INFO: `/v3/users/8a6d4325e3c14bfdabc4c1940d54bf6f` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:53.714638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6d4325e3c14bfdabc4c1940d54bf6f)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:53.718285 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:53.748185 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-f04d0b17-f7ba-465e-bd1a-c3a7417a52d0 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:53.749905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-f04d0b17-f7ba-465e-bd1a-c3a7417a52d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:53.761884 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1377/2753] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:53.765402 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d0b17-f7ba-465e-bd1a-c3a7417a52d0 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:53.776977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d0b17-f7ba-465e-bd1a-c3a7417a52d0 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:53.782901 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:53.783401 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:53.783679 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:53.783951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:53.784748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:06:53.784980 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:53.786640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:53.788982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6d4325e3c14bfdabc4c1940d54bf6f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.800326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1378/2754] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 16:06:53.804868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:53.806476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6d4325e3c14bfdabc4c1940d54bf6f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.816846 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655544c8-ed1c-42d4-ba55-8c1441c4f223 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:53.817909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6d4325e3c14bfdabc4c1940d54bf6f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.818536 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6d4325e3c14bfdabc4c1940d54bf6f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.822113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560ecbc-d9f3-47d8-8a4b-e70cf7f42c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:53.822797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560ecbc-d9f3-47d8-8a4b-e70cf7f42c30 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:53.823082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560ecbc-d9f3-47d8-8a4b-e70cf7f42c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:53.823363 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560ecbc-d9f3-47d8-8a4b-e70cf7f42c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:53.824069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560ecbc-d9f3-47d8-8a4b-e70cf7f42c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:53.826142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560ecbc-d9f3-47d8-8a4b-e70cf7f42c30 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:53.829353 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6d4325e3c14bfdabc4c1940d54bf6f'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:53.836965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1377/2755] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:53 2022] DELETE /identity/v3/users/8a6d4325e3c14bfdabc4c1940d54bf6f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:53.861821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1379/2756] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:53 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:06:53.864105 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:53.866865 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:53.946792 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fb6fd-4147-4339-b87b-4a03e2993d66 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.006915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:54.025420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322356e-2bd1-4d94-b631-2db784f4a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.025420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322356e-2bd1-4d94-b631-2db784f4a052 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.025420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322356e-2bd1-4d94-b631-2db784f4a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.025420 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322356e-2bd1-4d94-b631-2db784f4a052 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.025946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322356e-2bd1-4d94-b631-2db784f4a052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.027943 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322356e-2bd1-4d94-b631-2db784f4a052 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.053820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1378/2757] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:53 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:54.056041 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560ecbc-d9f3-47d8-8a4b-e70cf7f42c30 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.067148 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560ecbc-d9f3-47d8-8a4b-e70cf7f42c30 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.072038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ea33e-7826-4e2e-bee5-8dcdf5213960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.072723 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ea33e-7826-4e2e-bee5-8dcdf5213960 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.073004 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ea33e-7826-4e2e-bee5-8dcdf5213960 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.073278 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ea33e-7826-4e2e-bee5-8dcdf5213960 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.074046 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ea33e-7826-4e2e-bee5-8dcdf5213960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.075695 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ea33e-7826-4e2e-bee5-8dcdf5213960 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.087205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1380/2758] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:06:54.091761 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322356e-2bd1-4d94-b631-2db784f4a052 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.103091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322356e-2bd1-4d94-b631-2db784f4a052 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.108517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.109202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.109485 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.109762 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.110528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.112169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.123795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1379/2759] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:54.128102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ea33e-7826-4e2e-bee5-8dcdf5213960 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.139860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ea33e-7826-4e2e-bee5-8dcdf5213960 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.145355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b4d39-ca7d-465e-98bd-70ab8e9f2b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.146422 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b4d39-ca7d-465e-98bd-70ab8e9f2b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.146693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b4d39-ca7d-465e-98bd-70ab8e9f2b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.146971 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b4d39-ca7d-465e-98bd-70ab8e9f2b33 admin admin] PATH_INFO: `/v3/projects/4016a8a7626d4aa5a28d248a5a799e8e/users/8617fdc3e63446c89fcd22bd5756109d/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.162561 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.164562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:54.168142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b4d39-ca7d-465e-98bd-70ab8e9f2b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4016a8a7626d4aa5a28d248a5a799e8e, user_id=8617fdc3e63446c89fcd22bd5756109d, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.171362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b4d39-ca7d-465e-98bd-70ab8e9f2b33 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.180339 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fac94e-0478-4a9c-a592-ada754dde3c4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.186225 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.187123 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.187402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.187768 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] PATH_INFO: `/v3/projects/aa81e8477a3a45f7a27ffc308980456d` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.188910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa81e8477a3a45f7a27ffc308980456d)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.191021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.192301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1381/2760] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:54 2022] PUT /identity/v3/projects/4016a8a7626d4aa5a28d248a5a799e8e/users/8617fdc3e63446c89fcd22bd5756109d/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:54.197081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a29a75-dbb7-438b-a3af-ebf9dc41e470 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.197400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a29a75-dbb7-438b-a3af-ebf9dc41e470 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.197799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a29a75-dbb7-438b-a3af-ebf9dc41e470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.216620 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c1a29a75-dbb7-438b-a3af-ebf9dc41e470 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:54.218013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c1a29a75-dbb7-438b-a3af-ebf9dc41e470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:54.247411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa81e8477a3a45f7a27ffc308980456d', 'request_id': 'req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:54.258567 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa81e8477a3a45f7a27ffc308980456d', 'request_id': 'req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:54.267055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa81e8477a3a45f7a27ffc308980456d', 'request_id': 'req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:54.279480 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] The token cache is being invalidate because project aa81e8477a3a45f7a27ffc308980456d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:54.280559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:54.281221 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-1e2ba4a7-036d-48e6-ac3a-c45164bc4da2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:54.282922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1380/2761] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:54 2022] DELETE /identity/v3/projects/aa81e8477a3a45f7a27ffc308980456d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:54.287604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacdd08-77a4-47c1-bcbb-18f3e206d6f9 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.287900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacdd08-77a4-47c1-bcbb-18f3e206d6f9 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.288197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cacdd08-77a4-47c1-bcbb-18f3e206d6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.293741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1382/2762] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:54.303335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a29a75-dbb7-438b-a3af-ebf9dc41e470 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.304763 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c1a29a75-dbb7-438b-a3af-ebf9dc41e470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:54.309389 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-2cacdd08-77a4-47c1-bcbb-18f3e206d6f9 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:54.310659 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-2cacdd08-77a4-47c1-bcbb-18f3e206d6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:54.365070 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a29a75-dbb7-438b-a3af-ebf9dc41e470 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.373554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1381/2763] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:54.382322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7529b5-3c3d-4fb9-abe3-642d02cf6cb1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.382616 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7529b5-3c3d-4fb9-abe3-642d02cf6cb1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.382898 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7529b5-3c3d-4fb9-abe3-642d02cf6cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.394114 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:54.403839 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-3b7529b5-3c3d-4fb9-abe3-642d02cf6cb1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:54.405502 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-3b7529b5-3c3d-4fb9-abe3-642d02cf6cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:54.414405 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1382/2764] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:06:54.419033 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7529b5-3c3d-4fb9-abe3-642d02cf6cb1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.420925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa8db0-9bdf-4486-bd9a-504d27853d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.421611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa8db0-9bdf-4486-bd9a-504d27853d28 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.421932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa8db0-9bdf-4486-bd9a-504d27853d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.422203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa8db0-9bdf-4486-bd9a-504d27853d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.429603 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7529b5-3c3d-4fb9-abe3-642d02cf6cb1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.432568 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa8db0-9bdf-4486-bd9a-504d27853d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.434096 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa8db0-9bdf-4486-bd9a-504d27853d28 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.435208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.436415 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.436691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.436966 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.437732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:06:54.438097 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.439869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.446585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1383/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:06:54.457304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1383/2766] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 16:06:54.461992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa8db0-9bdf-4486-bd9a-504d27853d28 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.472977 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa8db0-9bdf-4486-bd9a-504d27853d28 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.481013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacf39-770e-4388-95f1-c08f324f4170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.481718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacf39-770e-4388-95f1-c08f324f4170 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.482898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacf39-770e-4388-95f1-c08f324f4170 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.483181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacf39-770e-4388-95f1-c08f324f4170 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.483896 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aacf39-770e-4388-95f1-c08f324f4170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.485907 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aacf39-770e-4388-95f1-c08f324f4170 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.515484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1384/2767] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:54 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:54.519752 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.534309 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc36b-7554-4645-90a9-a2438340bdb0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.566476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:54.605501 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2cb15-f81a-4cd0-9320-918b0d7fae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.607129 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2cb15-f81a-4cd0-9320-918b0d7fae0b admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.607414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2cb15-f81a-4cd0-9320-918b0d7fae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.607894 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2cb15-f81a-4cd0-9320-918b0d7fae0b admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.608702 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2cb15-f81a-4cd0-9320-918b0d7fae0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.610885 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2cb15-f81a-4cd0-9320-918b0d7fae0b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.636161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1384/2768] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:54 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:06:54.640667 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacf39-770e-4388-95f1-c08f324f4170 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.653289 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacf39-770e-4388-95f1-c08f324f4170 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.658886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc95326-62be-438c-99be-391b44f384ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.659590 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95326-62be-438c-99be-391b44f384ae admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.659882 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95326-62be-438c-99be-391b44f384ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.660603 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95326-62be-438c-99be-391b44f384ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.660923 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc95326-62be-438c-99be-391b44f384ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.662605 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc95326-62be-438c-99be-391b44f384ae admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.674418 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1385/2769] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:54.678782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2cb15-f81a-4cd0-9320-918b0d7fae0b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.690611 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2cb15-f81a-4cd0-9320-918b0d7fae0b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.696044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55a888-15d1-4183-8a43-f0888df99bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.697044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a888-15d1-4183-8a43-f0888df99bfe admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.697914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a888-15d1-4183-8a43-f0888df99bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.697914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a888-15d1-4183-8a43-f0888df99bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.698465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55a888-15d1-4183-8a43-f0888df99bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.700108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55a888-15d1-4183-8a43-f0888df99bfe admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.711789 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1385/2770] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:54.716727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc95326-62be-438c-99be-391b44f384ae admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:54.728395 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc95326-62be-438c-99be-391b44f384ae admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:54.735786 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100e9f-4b82-41ee-8a3a-dc15609d3f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:54.737343 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100e9f-4b82-41ee-8a3a-dc15609d3f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.737925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100e9f-4b82-41ee-8a3a-dc15609d3f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.738954 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58100e9f-4b82-41ee-8a3a-dc15609d3f8b admin admin] PATH_INFO: `/v3/projects/7b62433c1036488b9e1cf8f22f531957/users/d3c697e7e81d494c9d0acd9b81c940c2/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.765307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58100e9f-4b82-41ee-8a3a-dc15609d3f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b62433c1036488b9e1cf8f22f531957, user_id=d3c697e7e81d494c9d0acd9b81c940c2, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:54.768666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58100e9f-4b82-41ee-8a3a-dc15609d3f8b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:54.795334 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1386/2771] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:54 2022] PUT /identity/v3/projects/7b62433c1036488b9e1cf8f22f531957/users/d3c697e7e81d494c9d0acd9b81c940c2/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:54.809347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab431a4-da95-492b-9b87-91d0611639b3 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:54.809701 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab431a4-da95-492b-9b87-91d0611639b3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:54.810179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab431a4-da95-492b-9b87-91d0611639b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:54.841912 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-5ab431a4-da95-492b-9b87-91d0611639b3 None None] MFA Rules not processed for user `d3c697e7e81d494c9d0acd9b81c940c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:54.843497 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5ab431a4-da95-492b-9b87-91d0611639b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:54.926588 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1386/2772] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:54.931653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100e9f-4b82-41ee-8a3a-dc15609d3f8b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:55.000876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58100e9f-4b82-41ee-8a3a-dc15609d3f8b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:55.005686 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a32181-c1ac-4502-8c63-0eb44129f2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:55.006516 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a32181-c1ac-4502-8c63-0eb44129f2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.006663 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a32181-c1ac-4502-8c63-0eb44129f2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.006925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a32181-c1ac-4502-8c63-0eb44129f2f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.007621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a32181-c1ac-4502-8c63-0eb44129f2f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:55.009549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a32181-c1ac-4502-8c63-0eb44129f2f8 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:55.041344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1387/2773] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:54 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:55.042685 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab431a4-da95-492b-9b87-91d0611639b3 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:55.059111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab431a4-da95-492b-9b87-91d0611639b3 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:55.147825 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e6cd1-6aab-4b46-b75c-45daf1d61b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:55.148638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e6cd1-6aab-4b46-b75c-45daf1d61b05 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.149323 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e6cd1-6aab-4b46-b75c-45daf1d61b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.149676 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e6cd1-6aab-4b46-b75c-45daf1d61b05 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.150465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e6cd1-6aab-4b46-b75c-45daf1d61b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:55.152453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e6cd1-6aab-4b46-b75c-45daf1d61b05 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:55.181246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1387/2774] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:55 2022] POST /identity/v3/users => generated 300 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:55.182972 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a32181-c1ac-4502-8c63-0eb44129f2f8 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:55.195676 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a32181-c1ac-4502-8c63-0eb44129f2f8 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:55.202244 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc057c-e5a9-4be9-8afb-70cf5530f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:55.203406 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc057c-e5a9-4be9-8afb-70cf5530f5bc admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.203929 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc057c-e5a9-4be9-8afb-70cf5530f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.204670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc057c-e5a9-4be9-8afb-70cf5530f5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.206091 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc057c-e5a9-4be9-8afb-70cf5530f5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:55.209270 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc057c-e5a9-4be9-8afb-70cf5530f5bc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:55.227067 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1388/2775] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:55.231229 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e6cd1-6aab-4b46-b75c-45daf1d61b05 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:55.253013 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e6cd1-6aab-4b46-b75c-45daf1d61b05 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:55.258843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911b53-944c-4d00-a527-5db1a4900f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:55.260024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911b53-944c-4d00-a527-5db1a4900f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.260427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911b53-944c-4d00-a527-5db1a4900f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.260770 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de911b53-944c-4d00-a527-5db1a4900f5b admin admin] PATH_INFO: `/v3/projects/4e76618c106147a799ea28c8f34d4791/users/f20ac3aba75e49fa9b551b9173e99790/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.280679 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911b53-944c-4d00-a527-5db1a4900f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e76618c106147a799ea28c8f34d4791, user_id=f20ac3aba75e49fa9b551b9173e99790, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:55.284391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de911b53-944c-4d00-a527-5db1a4900f5b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:55.301274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1388/2776] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:55 2022] PUT /identity/v3/projects/4e76618c106147a799ea28c8f34d4791/users/f20ac3aba75e49fa9b551b9173e99790/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:55.305832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc057c-e5a9-4be9-8afb-70cf5530f5bc admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:55.316748 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc057c-e5a9-4be9-8afb-70cf5530f5bc admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:55.371561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:55.387795 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ba1d2-ae69-4156-a930-48d451fee6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:55.388610 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ba1d2-ae69-4156-a930-48d451fee6df admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.388973 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ba1d2-ae69-4156-a930-48d451fee6df admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.389631 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ba1d2-ae69-4156-a930-48d451fee6df admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.390773 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ba1d2-ae69-4156-a930-48d451fee6df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:55.393123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ba1d2-ae69-4156-a930-48d451fee6df admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:55.409042 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1389/2777] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:55.410861 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911b53-944c-4d00-a527-5db1a4900f5b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:55.418183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:55.424111 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de911b53-944c-4d00-a527-5db1a4900f5b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:55.430868 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:55.432390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.432692 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.432990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] PATH_INFO: `/v3/projects/4e76618c106147a799ea28c8f34d4791/users/f20ac3aba75e49fa9b551b9173e99790/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.435763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e76618c106147a799ea28c8f34d4791, user_id=f20ac3aba75e49fa9b551b9173e99790, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:55.438278 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:55.455684 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1389/2778] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:55 2022] PUT /identity/v3/projects/4e76618c106147a799ea28c8f34d4791/users/f20ac3aba75e49fa9b551b9173e99790/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:55.468380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4e223-b711-48f1-8857-302a7b16b4c1 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.468715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4e223-b711-48f1-8857-302a7b16b4c1 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.469103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4e223-b711-48f1-8857-302a7b16b4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.506708 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-fbd4e223-b711-48f1-8857-302a7b16b4c1 None None] MFA Rules not processed for user `f20ac3aba75e49fa9b551b9173e99790`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:55.507855 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-fbd4e223-b711-48f1-8857-302a7b16b4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:55.601158 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1390/2779] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:55.616963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:55.619119 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:55.693222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877c228-8d90-42eb-ac67-368c9a36eb38 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:55.698311 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84402f73-1fc8-448c-8938-28cf227cf093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:55.699082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402f73-1fc8-448c-8938-28cf227cf093 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.699435 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402f73-1fc8-448c-8938-28cf227cf093 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.699709 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402f73-1fc8-448c-8938-28cf227cf093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.711164 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84402f73-1fc8-448c-8938-28cf227cf093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:55.712704 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84402f73-1fc8-448c-8938-28cf227cf093 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:55.724481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1390/2780] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:55.766504 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4e223-b711-48f1-8857-302a7b16b4c1 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:55.842868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4e223-b711-48f1-8857-302a7b16b4c1 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:55.848794 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:55.849709 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.850035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.850312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] PATH_INFO: `/v3/users/d3c697e7e81d494c9d0acd9b81c940c2` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:55.851915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c697e7e81d494c9d0acd9b81c940c2)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:55.853916 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:55.921554 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c697e7e81d494c9d0acd9b81c940c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:55.935844 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c697e7e81d494c9d0acd9b81c940c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:55.945683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c697e7e81d494c9d0acd9b81c940c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:55.948712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:55.960765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c697e7e81d494c9d0acd9b81c940c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:55.961732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c697e7e81d494c9d0acd9b81c940c2'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:55.969153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1391/2781] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:55 2022] DELETE /identity/v3/users/d3c697e7e81d494c9d0acd9b81c940c2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:55.973836 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321eac-af06-4439-a9d6-dd89be21dc00 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:55.974249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321eac-af06-4439-a9d6-dd89be21dc00 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:55.974755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7321eac-af06-4439-a9d6-dd89be21dc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:56.001170 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-b7321eac-af06-4439-a9d6-dd89be21dc00 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:56.002424 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b7321eac-af06-4439-a9d6-dd89be21dc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:56.079979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1391/2782] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:56.103644 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:56.108510 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:56.198190 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23272b05-e4c1-43f3-9f99-689bf8b75448 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:56.204684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dd807-1dfe-4e53-b8a9-a569a76586bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:56.205398 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dd807-1dfe-4e53-b8a9-a569a76586bc service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:56.205746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dd807-1dfe-4e53-b8a9-a569a76586bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:56.207156 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8dd807-1dfe-4e53-b8a9-a569a76586bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:56.217959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8dd807-1dfe-4e53-b8a9-a569a76586bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:56.219459 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8dd807-1dfe-4e53-b8a9-a569a76586bc service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:56.230985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1392/2783] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:06:56.559049 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7321eac-af06-4439-a9d6-dd89be21dc00 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:56.560931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b7321eac-af06-4439-a9d6-dd89be21dc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:56.580599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7321eac-af06-4439-a9d6-dd89be21dc00 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:56.585634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:56.586937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:56.587143 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:56.587425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] PATH_INFO: `/v3/projects/7b62433c1036488b9e1cf8f22f531957` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:56.589399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b62433c1036488b9e1cf8f22f531957)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:56.591744 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:56.689345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b62433c1036488b9e1cf8f22f531957', 'request_id': 'req-08afb514-f3a1-46f0-af4e-2e8fea792d1b'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.712521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b62433c1036488b9e1cf8f22f531957', 'request_id': 'req-08afb514-f3a1-46f0-af4e-2e8fea792d1b'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.722636 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b62433c1036488b9e1cf8f22f531957', 'request_id': 'req-08afb514-f3a1-46f0-af4e-2e8fea792d1b'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.741712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] The token cache is being invalidate because project 7b62433c1036488b9e1cf8f22f531957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:56.742039 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.742751 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.744746 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1392/2784] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:56 2022] DELETE /identity/v3/projects/7b62433c1036488b9e1cf8f22f531957 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:56.749374 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dd807-1dfe-4e53-b8a9-a569a76586bc service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:56.768630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8dd807-1dfe-4e53-b8a9-a569a76586bc service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:56.847962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:56.849005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:56.849413 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:56.849876 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] PATH_INFO: `/v3/users/f20ac3aba75e49fa9b551b9173e99790` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:56.851768 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f20ac3aba75e49fa9b551b9173e99790)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:56.854259 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:56.925662 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f20ac3aba75e49fa9b551b9173e99790'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.947086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f20ac3aba75e49fa9b551b9173e99790'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.967134 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f20ac3aba75e49fa9b551b9173e99790'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.980013 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f20ac3aba75e49fa9b551b9173e99790'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.981305 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-8b76fc6b-8a62-4093-b322-c7af3979b1eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f20ac3aba75e49fa9b551b9173e99790'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:56.992005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1393/2785] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:06:56 2022] DELETE /identity/v3/users/f20ac3aba75e49fa9b551b9173e99790 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:57.554759 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:57.556452 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:57.576015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afb514-f3a1-46f0-af4e-2e8fea792d1b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:57.593243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:57.671381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:57.672315 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:57.672601 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:57.672952 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] PATH_INFO: `/v3/projects/4e76618c106147a799ea28c8f34d4791` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:57.675804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e76618c106147a799ea28c8f34d4791)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:57.678031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:57.748572 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e76618c106147a799ea28c8f34d4791', 'request_id': 'req-0083f643-fad3-433b-a51a-0de008c44641'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:57.758747 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e76618c106147a799ea28c8f34d4791', 'request_id': 'req-0083f643-fad3-433b-a51a-0de008c44641'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:57.767283 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e76618c106147a799ea28c8f34d4791', 'request_id': 'req-0083f643-fad3-433b-a51a-0de008c44641'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:57.788753 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] The token cache is being invalidate because project 4e76618c106147a799ea28c8f34d4791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:06:57.789212 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:57.790028 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-0083f643-fad3-433b-a51a-0de008c44641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:06:57.791955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1393/2786] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:06:57 2022] DELETE /identity/v3/projects/4e76618c106147a799ea28c8f34d4791 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:57.797103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e139fb-ff75-49fb-af3c-fa12bad1ed90 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:57.797979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e139fb-ff75-49fb-af3c-fa12bad1ed90 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:57.798262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e139fb-ff75-49fb-af3c-fa12bad1ed90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:57.816951 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-87e139fb-ff75-49fb-af3c-fa12bad1ed90 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:57.818506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-87e139fb-ff75-49fb-af3c-fa12bad1ed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:57.827421 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:57.933809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1394/2787] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:57.940558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a59c3-94cb-4c67-bf27-318b80976028 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:57.941203 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a59c3-94cb-4c67-bf27-318b80976028 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:57.941760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a59c3-94cb-4c67-bf27-318b80976028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:57.967297 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-406a59c3-94cb-4c67-bf27-318b80976028 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:57.968966 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-406a59c3-94cb-4c67-bf27-318b80976028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:57.989301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1394/2788] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:57.992981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e139fb-ff75-49fb-af3c-fa12bad1ed90 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.003909 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e139fb-ff75-49fb-af3c-fa12bad1ed90 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.008698 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.009376 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.010405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.010405 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.010903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:06:58.011214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.012859 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.031537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1395/2789] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 16:06:58.033439 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a59c3-94cb-4c67-bf27-318b80976028 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.045214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a59c3-94cb-4c67-bf27-318b80976028 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.050635 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366cd84-b690-4655-bf69-a06e303b02ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.051504 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366cd84-b690-4655-bf69-a06e303b02ee admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.052000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366cd84-b690-4655-bf69-a06e303b02ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.052000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366cd84-b690-4655-bf69-a06e303b02ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.052920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366cd84-b690-4655-bf69-a06e303b02ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.055216 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366cd84-b690-4655-bf69-a06e303b02ee admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.086567 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1395/2790] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:58 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:58.091078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.103585 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b289a54-7e64-4f7a-91d9-f4ad0b9255cd admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.186798 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b988de-3fae-4c9c-aa68-9b44a6acd3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.187801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b988de-3fae-4c9c-aa68-9b44a6acd3bf admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.188165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b988de-3fae-4c9c-aa68-9b44a6acd3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.188508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b988de-3fae-4c9c-aa68-9b44a6acd3bf admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.189636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b988de-3fae-4c9c-aa68-9b44a6acd3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.192037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b988de-3fae-4c9c-aa68-9b44a6acd3bf admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.224491 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366cd84-b690-4655-bf69-a06e303b02ee admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.233120 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1396/2791] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:58 2022] POST /identity/v3/users => generated 305 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:58.245647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366cd84-b690-4655-bf69-a06e303b02ee admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.252975 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b772-004b-4990-a6b9-ee2035fb5a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.257710 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b772-004b-4990-a6b9-ee2035fb5a48 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.258593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b772-004b-4990-a6b9-ee2035fb5a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.258973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b772-004b-4990-a6b9-ee2035fb5a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.260181 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b772-004b-4990-a6b9-ee2035fb5a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.262637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b772-004b-4990-a6b9-ee2035fb5a48 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.287265 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b988de-3fae-4c9c-aa68-9b44a6acd3bf admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.297324 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1396/2792] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:58.303155 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b988de-3fae-4c9c-aa68-9b44a6acd3bf admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.310221 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84e6ca-cac1-4d56-affe-a8269ec0c268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.311307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e6ca-cac1-4d56-affe-a8269ec0c268 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.311718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e6ca-cac1-4d56-affe-a8269ec0c268 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.312094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e6ca-cac1-4d56-affe-a8269ec0c268 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.312981 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84e6ca-cac1-4d56-affe-a8269ec0c268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.314765 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84e6ca-cac1-4d56-affe-a8269ec0c268 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.331957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1397/2793] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:58.336494 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b772-004b-4990-a6b9-ee2035fb5a48 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.349044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b772-004b-4990-a6b9-ee2035fb5a48 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.355021 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e5886-db3c-4c31-af9b-d4136bc35d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.356866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e5886-db3c-4c31-af9b-d4136bc35d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.357375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e5886-db3c-4c31-af9b-d4136bc35d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.357944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e5886-db3c-4c31-af9b-d4136bc35d25 admin admin] PATH_INFO: `/v3/projects/e8703bac8de146f48ef8c20979a80591/users/02dfa6ed906c45baa9c2fa18bb25d036/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.378594 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e5886-db3c-4c31-af9b-d4136bc35d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8703bac8de146f48ef8c20979a80591, user_id=02dfa6ed906c45baa9c2fa18bb25d036, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.386286 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e5886-db3c-4c31-af9b-d4136bc35d25 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.408565 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1397/2794] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:58 2022] PUT /identity/v3/projects/e8703bac8de146f48ef8c20979a80591/users/02dfa6ed906c45baa9c2fa18bb25d036/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:58.421760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4d24-b7e6-4516-8670-8f6d8c31f295 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.422170 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4d24-b7e6-4516-8670-8f6d8c31f295 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.422921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d4d24-b7e6-4516-8670-8f6d8c31f295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.453392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-cc9d4d24-b7e6-4516-8670-8f6d8c31f295 None None] MFA Rules not processed for user `02dfa6ed906c45baa9c2fa18bb25d036`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:58.454752 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-cc9d4d24-b7e6-4516-8670-8f6d8c31f295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:58.543933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1398/2795] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:58.548026 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e5886-db3c-4c31-af9b-d4136bc35d25 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.606115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:58.648648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e5886-db3c-4c31-af9b-d4136bc35d25 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.654940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6c071-85d2-4133-9115-98a30ce5c4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.655643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6c071-85d2-4133-9115-98a30ce5c4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.655930 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6c071-85d2-4133-9115-98a30ce5c4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.656391 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6c071-85d2-4133-9115-98a30ce5c4d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.657358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6c071-85d2-4133-9115-98a30ce5c4d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.661156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6c071-85d2-4133-9115-98a30ce5c4d8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.695990 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1398/2796] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:58 2022] POST /identity/v3/projects => generated 407 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:58.697429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d4d24-b7e6-4516-8670-8f6d8c31f295 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.710495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9d4d24-b7e6-4516-8670-8f6d8c31f295 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.718444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:58.819488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0bb44-7afd-4f7a-8307-674f5dfa1bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.822262 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0bb44-7afd-4f7a-8307-674f5dfa1bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.823454 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0bb44-7afd-4f7a-8307-674f5dfa1bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.824466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0bb44-7afd-4f7a-8307-674f5dfa1bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.831220 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0bb44-7afd-4f7a-8307-674f5dfa1bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.834961 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0bb44-7afd-4f7a-8307-674f5dfa1bd1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.890000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1399/2797] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:58 2022] POST /identity/v3/users => generated 310 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:06:58.893994 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6c071-85d2-4133-9115-98a30ce5c4d8 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.908544 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6c071-85d2-4133-9115-98a30ce5c4d8 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.921999 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0e754-338d-4063-b3d6-35fe8771e110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.922712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0e754-338d-4063-b3d6-35fe8771e110 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.922992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0e754-338d-4063-b3d6-35fe8771e110 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.923266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0e754-338d-4063-b3d6-35fe8771e110 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:58.924010 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0e754-338d-4063-b3d6-35fe8771e110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:58.930655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0e754-338d-4063-b3d6-35fe8771e110 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:58.947376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1399/2798] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:58.955682 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0bb44-7afd-4f7a-8307-674f5dfa1bd1 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:58.971762 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0bb44-7afd-4f7a-8307-674f5dfa1bd1 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:58.977921 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2731b0-75cb-4ace-92fa-9c3c24fc0cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:58.980895 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2731b0-75cb-4ace-92fa-9c3c24fc0cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:58.981396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2731b0-75cb-4ace-92fa-9c3c24fc0cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:58.982122 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2731b0-75cb-4ace-92fa-9c3c24fc0cf3 admin admin] PATH_INFO: `/v3/projects/b237717d8885407e9fb4e5e897c552fc/users/b85f24d7ab304ee08806e74c34e44999/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.003908 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2731b0-75cb-4ace-92fa-9c3c24fc0cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b237717d8885407e9fb4e5e897c552fc, user_id=b85f24d7ab304ee08806e74c34e44999, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.007277 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2731b0-75cb-4ace-92fa-9c3c24fc0cf3 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.033824 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1400/2799] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:58 2022] PUT /identity/v3/projects/b237717d8885407e9fb4e5e897c552fc/users/b85f24d7ab304ee08806e74c34e44999/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:59.036133 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0e754-338d-4063-b3d6-35fe8771e110 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.050335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0e754-338d-4063-b3d6-35fe8771e110 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.136684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b90a-5b0e-413f-b798-9f0c9a29b4c0 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.136982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b90a-5b0e-413f-b798-9f0c9a29b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.137252 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b90a-5b0e-413f-b798-9f0c9a29b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.156672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd5ad0-a7fa-4a99-a98b-4c82ba235689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.157867 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd5ad0-a7fa-4a99-a98b-4c82ba235689 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.158377 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd5ad0-a7fa-4a99-a98b-4c82ba235689 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.158914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd5ad0-a7fa-4a99-a98b-4c82ba235689 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.159766 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd5ad0-a7fa-4a99-a98b-4c82ba235689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.161889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd5ad0-a7fa-4a99-a98b-4c82ba235689 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.174756 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1400/2800] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:59.178757 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd5ad0-a7fa-4a99-a98b-4c82ba235689 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.187767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b2c3b90a-5b0e-413f-b798-9f0c9a29b4c0 None None] MFA Rules not processed for user `8617fdc3e63446c89fcd22bd5756109d`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:59.189117 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b2c3b90a-5b0e-413f-b798-9f0c9a29b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:59.191890 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd5ad0-a7fa-4a99-a98b-4c82ba235689 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.200221 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df73541-0a2c-4c27-b83b-01f1627d959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.201484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df73541-0a2c-4c27-b83b-01f1627d959e admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.201763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df73541-0a2c-4c27-b83b-01f1627d959e admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.202220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df73541-0a2c-4c27-b83b-01f1627d959e admin admin] PATH_INFO: `/v3/projects/b237717d8885407e9fb4e5e897c552fc/users/b85f24d7ab304ee08806e74c34e44999/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.204471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df73541-0a2c-4c27-b83b-01f1627d959e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b237717d8885407e9fb4e5e897c552fc, user_id=b85f24d7ab304ee08806e74c34e44999, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.207241 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df73541-0a2c-4c27-b83b-01f1627d959e admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.224222 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1401/2801] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:59 2022] PUT /identity/v3/projects/b237717d8885407e9fb4e5e897c552fc/users/b85f24d7ab304ee08806e74c34e44999/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:59.240508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a5f6a-9203-44a4-8c9f-cdeca87ec3cf None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.240804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a5f6a-9203-44a4-8c9f-cdeca87ec3cf None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.241114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a5f6a-9203-44a4-8c9f-cdeca87ec3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.284443 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-8a3a5f6a-9203-44a4-8c9f-cdeca87ec3cf None None] MFA Rules not processed for user `b85f24d7ab304ee08806e74c34e44999`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:06:59.285969 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8a3a5f6a-9203-44a4-8c9f-cdeca87ec3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:59.294879 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1401/2802] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:59.301825 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b90a-5b0e-413f-b798-9f0c9a29b4c0 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.303938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b2c3b90a-5b0e-413f-b798-9f0c9a29b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:59.326828 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b90a-5b0e-413f-b798-9f0c9a29b4c0 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.396511 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1402/2803] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:06:59.409664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a5f6a-9203-44a4-8c9f-cdeca87ec3cf None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.411220 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8a3a5f6a-9203-44a4-8c9f-cdeca87ec3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:06:59.419037 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b749c-6b7b-400e-bc5d-65627667ad7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.420340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b749c-6b7b-400e-bc5d-65627667ad7e admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.420621 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b749c-6b7b-400e-bc5d-65627667ad7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.420894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b749c-6b7b-400e-bc5d-65627667ad7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.421611 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b749c-6b7b-400e-bc5d-65627667ad7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.423718 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b749c-6b7b-400e-bc5d-65627667ad7e admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.429200 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:59.442169 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:59.461751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1402/2804] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:06:59 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:06:59.465829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b749c-6b7b-400e-bc5d-65627667ad7e admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.484066 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a5f6a-9203-44a4-8c9f-cdeca87ec3cf None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.543593 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812172b4-1af4-4506-9e9f-46d8c1beb816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.544308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812172b4-1af4-4506-9e9f-46d8c1beb816 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.545275 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812172b4-1af4-4506-9e9f-46d8c1beb816 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.545571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812172b4-1af4-4506-9e9f-46d8c1beb816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.545745 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b749c-6b7b-400e-bc5d-65627667ad7e admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.550868 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86524e8-9092-45e5-b54c-bfaddc856096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.552131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86524e8-9092-45e5-b54c-bfaddc856096 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.552131 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86524e8-9092-45e5-b54c-bfaddc856096 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.552325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86524e8-9092-45e5-b54c-bfaddc856096 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.552946 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86524e8-9092-45e5-b54c-bfaddc856096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.555014 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86524e8-9092-45e5-b54c-bfaddc856096 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.581217 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1403/2805] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:06:59 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:06:59.587123 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86524e8-9092-45e5-b54c-bfaddc856096 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.599263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86524e8-9092-45e5-b54c-bfaddc856096 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.605140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40467015-08ff-48eb-8919-6baeb34464f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.606110 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467015-08ff-48eb-8919-6baeb34464f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.606578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467015-08ff-48eb-8919-6baeb34464f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.607711 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467015-08ff-48eb-8919-6baeb34464f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.609257 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40467015-08ff-48eb-8919-6baeb34464f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.611697 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40467015-08ff-48eb-8919-6baeb34464f4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.622729 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812172b4-1af4-4506-9e9f-46d8c1beb816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.624985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812172b4-1af4-4506-9e9f-46d8c1beb816 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.628301 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1404/2806] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:06:59.633233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40467015-08ff-48eb-8919-6baeb34464f4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.636466 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1403/2807] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:06:59.648935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40467015-08ff-48eb-8919-6baeb34464f4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.654325 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea606f-c68c-451f-a4b4-3e47f7ff5455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.655444 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea606f-c68c-451f-a4b4-3e47f7ff5455 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.655774 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea606f-c68c-451f-a4b4-3e47f7ff5455 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.656258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea606f-c68c-451f-a4b4-3e47f7ff5455 admin admin] PATH_INFO: `/v3/projects/4e019b6e83324155848189e56bdf0a4e/users/c675841a37b5419aa6d448cfa90624ed/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.679140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea606f-c68c-451f-a4b4-3e47f7ff5455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e019b6e83324155848189e56bdf0a4e, user_id=c675841a37b5419aa6d448cfa90624ed, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.683462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea606f-c68c-451f-a4b4-3e47f7ff5455 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.700821 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1405/2808] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:59 2022] PUT /identity/v3/projects/4e019b6e83324155848189e56bdf0a4e/users/c675841a37b5419aa6d448cfa90624ed/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:06:59.705175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812172b4-1af4-4506-9e9f-46d8c1beb816 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.718619 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812172b4-1af4-4506-9e9f-46d8c1beb816 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.787183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19714205-736a-4235-ae5b-264f85f466c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.787960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19714205-736a-4235-ae5b-264f85f466c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.788250 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19714205-736a-4235-ae5b-264f85f466c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.788528 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19714205-736a-4235-ae5b-264f85f466c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.789271 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19714205-736a-4235-ae5b-264f85f466c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.791201 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19714205-736a-4235-ae5b-264f85f466c0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.801376 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1404/2809] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:06:59.809561 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea606f-c68c-451f-a4b4-3e47f7ff5455 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.813703 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19714205-736a-4235-ae5b-264f85f466c0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:06:59.820688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea606f-c68c-451f-a4b4-3e47f7ff5455 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.826003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c187f9-212d-4078-adc4-b112b5be76b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.826913 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c187f9-212d-4078-adc4-b112b5be76b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.827193 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c187f9-212d-4078-adc4-b112b5be76b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.827466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c187f9-212d-4078-adc4-b112b5be76b5 admin admin] PATH_INFO: `/v3/projects/4e019b6e83324155848189e56bdf0a4e/users/c675841a37b5419aa6d448cfa90624ed/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:06:59.829466 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c187f9-212d-4078-adc4-b112b5be76b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e019b6e83324155848189e56bdf0a4e, user_id=c675841a37b5419aa6d448cfa90624ed, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:06:59.832174 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c187f9-212d-4078-adc4-b112b5be76b5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:06:59.843964 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1406/2810] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:06:59 2022] PUT /identity/v3/projects/4e019b6e83324155848189e56bdf0a4e/users/c675841a37b5419aa6d448cfa90624ed/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:06:59.846011 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:06:59.879370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19714205-736a-4235-ae5b-264f85f466c0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:06:59.935918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:06:59.936803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:06:59.936960 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:06:59.937244 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:00.014817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:00.017024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:00.031469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1405/2811] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:07:04.352577 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2fe51-f27b-49ce-a931-6205526db0ec None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:04.353488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2fe51-f27b-49ce-a931-6205526db0ec None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:04.354000 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2fe51-f27b-49ce-a931-6205526db0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:04.395307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-38a2fe51-f27b-49ce-a931-6205526db0ec None None] MFA Rules not processed for user `c675841a37b5419aa6d448cfa90624ed`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:04.400949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-38a2fe51-f27b-49ce-a931-6205526db0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:04.518785 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1407/2812] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:07:04.536642 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:04.538282 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:04.633646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65bb22-b51a-4d4f-bc26-a683c83b1f70 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:04.639694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74d17d-a6c6-4257-9bc1-027af16316ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:04.640362 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74d17d-a6c6-4257-9bc1-027af16316ca service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:04.640644 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74d17d-a6c6-4257-9bc1-027af16316ca service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:04.641023 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74d17d-a6c6-4257-9bc1-027af16316ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:04.703558 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=100 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:04.741936 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74d17d-a6c6-4257-9bc1-027af16316ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:04.743707 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74d17d-a6c6-4257-9bc1-027af16316ca service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:04.757073 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1406/2813] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:05.840273 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793d0a1-3593-418f-a3f3-d53cf96b1a1c None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:05.840788 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793d0a1-3593-418f-a3f3-d53cf96b1a1c None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:05.840978 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793d0a1-3593-418f-a3f3-d53cf96b1a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:05.861101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-5793d0a1-3593-418f-a3f3-d53cf96b1a1c None None] MFA Rules not processed for user `8617fdc3e63446c89fcd22bd5756109d`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:05.862446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-5793d0a1-3593-418f-a3f3-d53cf96b1a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:05.871029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1408/2814] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:08.765499 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74d17d-a6c6-4257-9bc1-027af16316ca service nova] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:08.873274 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74d17d-a6c6-4257-9bc1-027af16316ca service nova] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:08.881292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:08.882380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:08.882664 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:08.882942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] PATH_INFO: `/v3/users/02dfa6ed906c45baa9c2fa18bb25d036` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:08.884421 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02dfa6ed906c45baa9c2fa18bb25d036)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:08.886649 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:08.961094 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02dfa6ed906c45baa9c2fa18bb25d036'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:08.975914 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02dfa6ed906c45baa9c2fa18bb25d036'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:08.986925 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02dfa6ed906c45baa9c2fa18bb25d036'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:08.987541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02dfa6ed906c45baa9c2fa18bb25d036'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:08.996918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02dfa6ed906c45baa9c2fa18bb25d036'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.007029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1407/2815] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:07:08 2022] DELETE /identity/v3/users/02dfa6ed906c45baa9c2fa18bb25d036 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:09.011797 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63448746-d942-4fa1-a31b-c046930f1373 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:09.012539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63448746-d942-4fa1-a31b-c046930f1373 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:09.012539 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63448746-d942-4fa1-a31b-c046930f1373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:09.025822 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:09.035462 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-63448746-d942-4fa1-a31b-c046930f1373 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:09.037886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-63448746-d942-4fa1-a31b-c046930f1373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:09.110291 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1409/2816] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:09.120370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:09.122357 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:09.189375 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:09.200081 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50dc5b-5611-4a04-b944-56ab00105df4 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:09.206004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5b1f-0650-4b98-a701-5ec4defee435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:09.207171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5b1f-0650-4b98-a701-5ec4defee435 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:09.213031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5b1f-0650-4b98-a701-5ec4defee435 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:09.213328 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5b1f-0650-4b98-a701-5ec4defee435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:09.225564 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a5b1f-0650-4b98-a701-5ec4defee435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:09.227463 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a5b1f-0650-4b98-a701-5ec4defee435 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:09.238794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1408/2817] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:09.563452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63448746-d942-4fa1-a31b-c046930f1373 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:09.564963 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-63448746-d942-4fa1-a31b-c046930f1373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:09.578054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63448746-d942-4fa1-a31b-c046930f1373 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:09.582979 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:09.583873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:09.584224 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:09.584508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] PATH_INFO: `/v3/projects/e8703bac8de146f48ef8c20979a80591` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:09.585760 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8703bac8de146f48ef8c20979a80591)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:09.587729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:09.640722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8703bac8de146f48ef8c20979a80591', 'request_id': 'req-91cebbde-32be-4659-8d19-21050edd1966'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.649488 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8703bac8de146f48ef8c20979a80591', 'request_id': 'req-91cebbde-32be-4659-8d19-21050edd1966'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.658617 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8703bac8de146f48ef8c20979a80591', 'request_id': 'req-91cebbde-32be-4659-8d19-21050edd1966'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.671160 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] The token cache is being invalidate because project e8703bac8de146f48ef8c20979a80591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:07:09.671469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.672154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.674005 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1410/2818] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:07:09 2022] DELETE /identity/v3/projects/e8703bac8de146f48ef8c20979a80591 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:09.678159 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5b1f-0650-4b98-a701-5ec4defee435 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:09.688974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5b1f-0650-4b98-a701-5ec4defee435 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:09.752889 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:09.753922 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:09.754215 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:09.755522 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] PATH_INFO: `/v3/users/b85f24d7ab304ee08806e74c34e44999` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:09.756938 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85f24d7ab304ee08806e74c34e44999)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:09.758937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:09.812947 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85f24d7ab304ee08806e74c34e44999'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.826996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85f24d7ab304ee08806e74c34e44999'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.836838 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85f24d7ab304ee08806e74c34e44999'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.837634 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85f24d7ab304ee08806e74c34e44999'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.846154 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-826a3667-29b2-4b15-b009-02e9e85d8090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85f24d7ab304ee08806e74c34e44999'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:09.853243 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1409/2819] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:07:09 2022] DELETE /identity/v3/users/b85f24d7ab304ee08806e74c34e44999 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:10.153728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:10.157591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:10.172670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cebbde-32be-4659-8d19-21050edd1966 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:10.252375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:10.253285 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.253566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.253873 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] PATH_INFO: `/v3/projects/b237717d8885407e9fb4e5e897c552fc` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.255166 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b237717d8885407e9fb4e5e897c552fc)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:10.257214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:10.300887 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:10.316717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b237717d8885407e9fb4e5e897c552fc', 'request_id': 'req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:10.323992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b237717d8885407e9fb4e5e897c552fc', 'request_id': 'req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:10.335818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b237717d8885407e9fb4e5e897c552fc', 'request_id': 'req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:10.349970 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] The token cache is being invalidate because project b237717d8885407e9fb4e5e897c552fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:07:10.350317 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:10.350982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e8ce0874-b8f3-4608-bfbf-5370d4b49c4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:10.353147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1411/2820] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:07:10 2022] DELETE /identity/v3/projects/b237717d8885407e9fb4e5e897c552fc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:10.358226 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9af66-9ff5-440f-ae4c-dd48d60d433c None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.358646 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9af66-9ff5-440f-ae4c-dd48d60d433c None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.358980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9af66-9ff5-440f-ae4c-dd48d60d433c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.378232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-23f9af66-9ff5-440f-ae4c-dd48d60d433c None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:10.379441 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-23f9af66-9ff5-440f-ae4c-dd48d60d433c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:10.444506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1410/2821] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:10.452689 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed87507-48dc-4781-b6e9-4e28ef166e26 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.452969 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed87507-48dc-4781-b6e9-4e28ef166e26 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.453248 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed87507-48dc-4781-b6e9-4e28ef166e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.471727 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1ed87507-48dc-4781-b6e9-4e28ef166e26 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:10.472935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1ed87507-48dc-4781-b6e9-4e28ef166e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:10.481543 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1412/2822] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:10.486179 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9af66-9ff5-440f-ae4c-dd48d60d433c None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:10.498412 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9af66-9ff5-440f-ae4c-dd48d60d433c None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:10.503950 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:10.504638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.504923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.505202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.505988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:07:10.506299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:10.507963 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:10.510663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:10.522458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1411/2823] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:07:10.527159 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed87507-48dc-4781-b6e9-4e28ef166e26 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:10.537008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed87507-48dc-4781-b6e9-4e28ef166e26 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:10.542985 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9558ab-cf8c-4f1f-85f6-22ae65dca00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:10.543655 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9558ab-cf8c-4f1f-85f6-22ae65dca00f admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.543935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9558ab-cf8c-4f1f-85f6-22ae65dca00f admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.544212 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9558ab-cf8c-4f1f-85f6-22ae65dca00f admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.545039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9558ab-cf8c-4f1f-85f6-22ae65dca00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:10.547071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9558ab-cf8c-4f1f-85f6-22ae65dca00f admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:10.578205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1413/2824] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:07:10 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:10.579477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:10.589881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6c727-f366-4536-bf55-c52455b2a994 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:10.656900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309d6c3-d75f-47f9-a9f8-7febc69a4029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:10.657599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d6c3-d75f-47f9-a9f8-7febc69a4029 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.657916 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d6c3-d75f-47f9-a9f8-7febc69a4029 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.658196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309d6c3-d75f-47f9-a9f8-7febc69a4029 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.658909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309d6c3-d75f-47f9-a9f8-7febc69a4029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:10.660862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309d6c3-d75f-47f9-a9f8-7febc69a4029 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:10.682650 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1412/2825] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:07:10 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:10.687206 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9558ab-cf8c-4f1f-85f6-22ae65dca00f admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:10.698035 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9558ab-cf8c-4f1f-85f6-22ae65dca00f admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:10.703751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1f862-6fa0-4628-9b87-cce7be9917d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:10.704490 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1f862-6fa0-4628-9b87-cce7be9917d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.704812 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1f862-6fa0-4628-9b87-cce7be9917d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.705108 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1f862-6fa0-4628-9b87-cce7be9917d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.705897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1f862-6fa0-4628-9b87-cce7be9917d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:10.708085 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1f862-6fa0-4628-9b87-cce7be9917d7 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:10.722113 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1414/2826] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:07:10.726822 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309d6c3-d75f-47f9-a9f8-7febc69a4029 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:10.738455 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309d6c3-d75f-47f9-a9f8-7febc69a4029 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:10.744099 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eef47a-0a68-49fd-a171-367c08cbca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:10.744765 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eef47a-0a68-49fd-a171-367c08cbca7f admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.745034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eef47a-0a68-49fd-a171-367c08cbca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.745346 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eef47a-0a68-49fd-a171-367c08cbca7f admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.746183 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eef47a-0a68-49fd-a171-367c08cbca7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:10.747795 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eef47a-0a68-49fd-a171-367c08cbca7f admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:10.758892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1413/2827] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:07:10.763271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1f862-6fa0-4628-9b87-cce7be9917d7 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:10.773819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1f862-6fa0-4628-9b87-cce7be9917d7 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:10.778591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:10.779606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.779890 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.780192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] PATH_INFO: `/v3/projects/1296511bc4564c208a7a263e8773470d/users/bb6c4aa411da45c59897cfe0913a7584/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.799203 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1296511bc4564c208a7a263e8773470d, user_id=bb6c4aa411da45c59897cfe0913a7584, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:10.802104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:10.817355 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1415/2828] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:07:10 2022] PUT /identity/v3/projects/1296511bc4564c208a7a263e8773470d/users/bb6c4aa411da45c59897cfe0913a7584/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:10.822919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65664ab1-361e-4d1d-a9f9-eb9c63eb2d8e None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.823476 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65664ab1-361e-4d1d-a9f9-eb9c63eb2d8e None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.823577 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65664ab1-361e-4d1d-a9f9-eb9c63eb2d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:10.842240 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-65664ab1-361e-4d1d-a9f9-eb9c63eb2d8e None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:10.843465 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-65664ab1-361e-4d1d-a9f9-eb9c63eb2d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:10.909032 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1414/2829] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:10.919029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:10.921508 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:10.990053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da869a-9865-4902-bd75-0d3c717f047b admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:10.993250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c658e9-50af-4f6a-8bb5-f09be8158593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:10.993966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c658e9-50af-4f6a-8bb5-f09be8158593 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:10.994924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c658e9-50af-4f6a-8bb5-f09be8158593 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:10.995213 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c658e9-50af-4f6a-8bb5-f09be8158593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:11.006375 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c658e9-50af-4f6a-8bb5-f09be8158593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:11.007905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c658e9-50af-4f6a-8bb5-f09be8158593 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:11.019960 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1416/2830] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:11.690139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755a532-b7bf-43f2-8e85-542d40033cd1 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:11.690477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755a532-b7bf-43f2-8e85-542d40033cd1 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:11.690760 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755a532-b7bf-43f2-8e85-542d40033cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:11.730115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-c755a532-b7bf-43f2-8e85-542d40033cd1 None None] MFA Rules not processed for user `bb6c4aa411da45c59897cfe0913a7584`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:11.731387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c755a532-b7bf-43f2-8e85-542d40033cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:11.799453 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1415/2831] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:11.806002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c658e9-50af-4f6a-8bb5-f09be8158593 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:11.816915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c658e9-50af-4f6a-8bb5-f09be8158593 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:11.821815 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33452a-839a-48d2-b20d-7c9623f41c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:11.822501 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33452a-839a-48d2-b20d-7c9623f41c83 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:11.822776 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33452a-839a-48d2-b20d-7c9623f41c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:11.823054 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33452a-839a-48d2-b20d-7c9623f41c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:11.823749 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33452a-839a-48d2-b20d-7c9623f41c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:11.825706 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33452a-839a-48d2-b20d-7c9623f41c83 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:11.870790 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1417/2832] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:07:11 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 21 16:07:11.875350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755a532-b7bf-43f2-8e85-542d40033cd1 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:11.882213 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:11.886631 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755a532-b7bf-43f2-8e85-542d40033cd1 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:11.946695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780255e-895b-497a-96c5-3413c22e15ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:11.947370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780255e-895b-497a-96c5-3413c22e15ec admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:11.947651 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780255e-895b-497a-96c5-3413c22e15ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:11.947931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780255e-895b-497a-96c5-3413c22e15ec admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:11.948632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780255e-895b-497a-96c5-3413c22e15ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:11.950748 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780255e-895b-497a-96c5-3413c22e15ec admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:11.972134 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1416/2833] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:07:11 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:11.976194 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33452a-839a-48d2-b20d-7c9623f41c83 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:11.988116 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33452a-839a-48d2-b20d-7c9623f41c83 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:11.994126 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18250175-df02-4bdb-bb60-4cdf2dadf8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:11.994811 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250175-df02-4bdb-bb60-4cdf2dadf8dc admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:11.995147 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250175-df02-4bdb-bb60-4cdf2dadf8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:11.995792 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18250175-df02-4bdb-bb60-4cdf2dadf8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:11.996637 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18250175-df02-4bdb-bb60-4cdf2dadf8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:11.998428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18250175-df02-4bdb-bb60-4cdf2dadf8dc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:12.012211 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1418/2834] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:07:12.017114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780255e-895b-497a-96c5-3413c22e15ec admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:12.028171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780255e-895b-497a-96c5-3413c22e15ec admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:12.034399 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed3436-7df1-4ae3-af66-2dcbac295222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:12.035312 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed3436-7df1-4ae3-af66-2dcbac295222 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:12.035600 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed3436-7df1-4ae3-af66-2dcbac295222 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:12.035906 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed3436-7df1-4ae3-af66-2dcbac295222 admin admin] PATH_INFO: `/v3/projects/aa8ace6d7cbf4e7783739e4f5c9c9379/users/87ef94dded914bf29f079f8ec5ffd8fb/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:12.052059 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed3436-7df1-4ae3-af66-2dcbac295222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8ace6d7cbf4e7783739e4f5c9c9379, user_id=87ef94dded914bf29f079f8ec5ffd8fb, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:12.054902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed3436-7df1-4ae3-af66-2dcbac295222 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:12.069156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1417/2835] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:07:12 2022] PUT /identity/v3/projects/aa8ace6d7cbf4e7783739e4f5c9c9379/users/87ef94dded914bf29f079f8ec5ffd8fb/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:12.073140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18250175-df02-4bdb-bb60-4cdf2dadf8dc admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:12.084558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18250175-df02-4bdb-bb60-4cdf2dadf8dc admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:12.113331 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:12.150094 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73572cd3-9646-4a8e-8aa5-991b181f0f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:12.150792 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572cd3-9646-4a8e-8aa5-991b181f0f60 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:12.151075 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572cd3-9646-4a8e-8aa5-991b181f0f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:12.151350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572cd3-9646-4a8e-8aa5-991b181f0f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:12.152087 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73572cd3-9646-4a8e-8aa5-991b181f0f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:12.153832 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73572cd3-9646-4a8e-8aa5-991b181f0f60 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:12.167653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1419/2836] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:07:12.172471 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed3436-7df1-4ae3-af66-2dcbac295222 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:12.183985 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed3436-7df1-4ae3-af66-2dcbac295222 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:12.189875 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:12.190782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:12.191075 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:12.191348 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] PATH_INFO: `/v3/projects/aa8ace6d7cbf4e7783739e4f5c9c9379/users/87ef94dded914bf29f079f8ec5ffd8fb/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:12.193381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8ace6d7cbf4e7783739e4f5c9c9379, user_id=87ef94dded914bf29f079f8ec5ffd8fb, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:12.196093 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:12.207392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1418/2837] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:07:12 2022] PUT /identity/v3/projects/aa8ace6d7cbf4e7783739e4f5c9c9379/users/87ef94dded914bf29f079f8ec5ffd8fb/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:13.106321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c4fe-d88e-4cbb-8766-7e77aab4cbbb None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:13.106321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c4fe-d88e-4cbb-8766-7e77aab4cbbb None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:13.106984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037c4fe-d88e-4cbb-8766-7e77aab4cbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:13.136900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-3037c4fe-d88e-4cbb-8766-7e77aab4cbbb None None] MFA Rules not processed for user `87ef94dded914bf29f079f8ec5ffd8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:13.138261 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3037c4fe-d88e-4cbb-8766-7e77aab4cbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:13.227880 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1420/2838] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:13.244991 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:13.247428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:13.316630 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aead54-079d-4a29-ab4c-225247b93e2d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:13.320656 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:13.321305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:13.321654 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:13.322020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:13.386776 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:13.388869 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:13.398301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1419/2839] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:14.787843 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037c4fe-d88e-4cbb-8766-7e77aab4cbbb None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:14.789715 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3037c4fe-d88e-4cbb-8766-7e77aab4cbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:14.803842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037c4fe-d88e-4cbb-8766-7e77aab4cbbb None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:14.807886 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:14.808636 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:14.809002 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:14.809434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:14.819905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:14.821372 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:14.831684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1421/2840] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:23.394249 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:24.827026 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=57 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:27.785897 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:27.787850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:27.852782 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0741939-2345-4836-a14f-61515fa5eca6 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:27.859927 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:27.860964 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:27.861633 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:27.862108 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:27.865031 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:27.876005 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:27.877647 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:27.886196 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1420/2841] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:28.378218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:28.380247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:28.466271 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb3090-ea97-49d6-be4e-787450e4aad9 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:28.473099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:28.473312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:28.473606 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:28.473898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:28.475713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:28.489535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:28.491306 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:28.505990 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1422/2842] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:07:29.372202 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:29.477343 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687766a-f5ed-4112-9316-c25c79f64f33 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:29.484518 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:29.485434 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:29.485717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:29.486020 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] PATH_INFO: `/v3/users/bb6c4aa411da45c59897cfe0913a7584` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:29.487909 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6c4aa411da45c59897cfe0913a7584)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:29.491427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:29.548060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6c4aa411da45c59897cfe0913a7584'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:29.570506 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6c4aa411da45c59897cfe0913a7584'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:29.588974 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6c4aa411da45c59897cfe0913a7584'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:29.589820 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6c4aa411da45c59897cfe0913a7584'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:29.602340 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6c4aa411da45c59897cfe0913a7584'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:29.611946 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1421/2843] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:07:29 2022] DELETE /identity/v3/users/bb6c4aa411da45c59897cfe0913a7584 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:29.817802 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:29.836799 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1df127-e556-4e86-8fab-98196ab739f3 service placement] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:29.934392 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:29.935704 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:29.936222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:29.936588 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] PATH_INFO: `/v3/projects/1296511bc4564c208a7a263e8773470d` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:29.942520 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1296511bc4564c208a7a263e8773470d)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:29.945322 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:30.021842 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1296511bc4564c208a7a263e8773470d', 'request_id': 'req-3b9d677e-defb-40c9-a6f6-206e8ad8d288'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.029064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1296511bc4564c208a7a263e8773470d', 'request_id': 'req-3b9d677e-defb-40c9-a6f6-206e8ad8d288'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.038959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1296511bc4564c208a7a263e8773470d', 'request_id': 'req-3b9d677e-defb-40c9-a6f6-206e8ad8d288'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.052064 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] The token cache is being invalidate because project 1296511bc4564c208a7a263e8773470d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:07:30.052351 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.053101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.054739 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1423/2844] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:07:29 2022] DELETE /identity/v3/projects/1296511bc4564c208a7a263e8773470d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:30.058945 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:30.061158 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:30.077069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182df3de-e438-4551-9a75-80d94178d90b admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:30.104336 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:30.164539 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:30.165638 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:30.166037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:30.166037 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] PATH_INFO: `/v3/users/87ef94dded914bf29f079f8ec5ffd8fb` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:30.167251 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ef94dded914bf29f079f8ec5ffd8fb)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:30.169187 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:30.226982 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ef94dded914bf29f079f8ec5ffd8fb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.242092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ef94dded914bf29f079f8ec5ffd8fb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.252918 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ef94dded914bf29f079f8ec5ffd8fb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.253718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ef94dded914bf29f079f8ec5ffd8fb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.262058 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-024db33f-f8c8-4e6a-b445-b6a1ecbc7be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ef94dded914bf29f079f8ec5ffd8fb'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.268449 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1422/2845] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:07:30 2022] DELETE /identity/v3/users/87ef94dded914bf29f079f8ec5ffd8fb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:07:30.487614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:30.489153 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:30.515751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d677e-defb-40c9-a6f6-206e8ad8d288 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:30.540630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:30.606099 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:30.606994 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:30.607387 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:30.607574 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] PATH_INFO: `/v3/projects/aa8ace6d7cbf4e7783739e4f5c9c9379` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:30.608888 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8ace6d7cbf4e7783739e4f5c9c9379)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:30.610852 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:30.683531 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8ace6d7cbf4e7783739e4f5c9c9379', 'request_id': 'req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.692204 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8ace6d7cbf4e7783739e4f5c9c9379', 'request_id': 'req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.707782 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8ace6d7cbf4e7783739e4f5c9c9379', 'request_id': 'req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.722633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] The token cache is being invalidate because project aa8ace6d7cbf4e7783739e4f5c9c9379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:07:30.722915 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.723619 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-bd55dc1f-f2aa-41ab-bf18-17d69b3859c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:30.725443 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1424/2846] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:07:30 2022] DELETE /identity/v3/projects/aa8ace6d7cbf4e7783739e4f5c9c9379 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:30.731004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524209a-9d38-4af4-846c-f92dde3103c3 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:30.731548 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524209a-9d38-4af4-846c-f92dde3103c3 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:30.731655 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524209a-9d38-4af4-846c-f92dde3103c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:30.751524 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-8524209a-9d38-4af4-846c-f92dde3103c3 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:30.753171 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8524209a-9d38-4af4-846c-f92dde3103c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:30.841082 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1423/2847] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:07:30.853192 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fd20-25e8-4760-910f-27cc2ff8d7c7 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:30.853721 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fd20-25e8-4760-910f-27cc2ff8d7c7 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:30.853992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365fd20-25e8-4760-910f-27cc2ff8d7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:30.883589 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-8365fd20-25e8-4760-910f-27cc2ff8d7c7 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:30.884856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-8365fd20-25e8-4760-910f-27cc2ff8d7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:30.897340 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1425/2848] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:30.901368 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524209a-9d38-4af4-846c-f92dde3103c3 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:30.912477 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524209a-9d38-4af4-846c-f92dde3103c3 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:30.917554 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:30.918366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:30.918794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:30.919139 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] PATH_INFO: `/v3/domains` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:30.919957 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86658) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:07:30.920305 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:30.922046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:30.939843 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1424/2849] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 21 16:07:30.941881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365fd20-25e8-4760-910f-27cc2ff8d7c7 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:30.953353 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365fd20-25e8-4760-910f-27cc2ff8d7c7 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:30.958506 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ceb39-6d93-4e8c-b32d-b8bc4c096634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:30.959218 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ceb39-6d93-4e8c-b32d-b8bc4c096634 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:30.959639 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ceb39-6d93-4e8c-b32d-b8bc4c096634 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:30.959914 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ceb39-6d93-4e8c-b32d-b8bc4c096634 admin admin] PATH_INFO: `/v3/projects` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:30.960653 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ceb39-6d93-4e8c-b32d-b8bc4c096634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:30.962809 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ceb39-6d93-4e8c-b32d-b8bc4c096634 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:30.997841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1426/2850] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:07:30 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:31.003051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:31.014160 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fba81-24f1-4631-87c6-39af717add54 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:31.048718 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:31.087043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d9805-c505-4ca7-93c3-9347c8ec04b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:31.087727 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d9805-c505-4ca7-93c3-9347c8ec04b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.088029 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d9805-c505-4ca7-93c3-9347c8ec04b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.088281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d9805-c505-4ca7-93c3-9347c8ec04b3 admin admin] PATH_INFO: `/v3/users` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.088979 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d9805-c505-4ca7-93c3-9347c8ec04b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:31.091291 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d9805-c505-4ca7-93c3-9347c8ec04b3 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:31.119044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1425/2851] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:07:31 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:31.120321 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ceb39-6d93-4e8c-b32d-b8bc4c096634 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:31.132837 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ceb39-6d93-4e8c-b32d-b8bc4c096634 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:31.137801 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccefc6e-e8df-4144-b306-18c0bcf48bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:31.138984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccefc6e-e8df-4144-b306-18c0bcf48bfc admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.138984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccefc6e-e8df-4144-b306-18c0bcf48bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.139214 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccefc6e-e8df-4144-b306-18c0bcf48bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.139808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccefc6e-e8df-4144-b306-18c0bcf48bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:31.141442 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccefc6e-e8df-4144-b306-18c0bcf48bfc admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:31.154071 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1427/2852] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:07:31.158618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d9805-c505-4ca7-93c3-9347c8ec04b3 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:31.174425 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d9805-c505-4ca7-93c3-9347c8ec04b3 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:31.179942 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9604192-fdb8-4279-a7b6-060f7f711713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:31.180887 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9604192-fdb8-4279-a7b6-060f7f711713 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.181167 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9604192-fdb8-4279-a7b6-060f7f711713 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.181613 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9604192-fdb8-4279-a7b6-060f7f711713 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.182247 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9604192-fdb8-4279-a7b6-060f7f711713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:31.183901 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9604192-fdb8-4279-a7b6-060f7f711713 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:31.199845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1426/2853] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:07:31.204171 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccefc6e-e8df-4144-b306-18c0bcf48bfc admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:31.215959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccefc6e-e8df-4144-b306-18c0bcf48bfc admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:31.221999 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:31.222922 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.223205 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.223484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] PATH_INFO: `/v3/projects/768d7e1ac8b24a41a1fe64761eb5249b/users/1e9c7afa908c4d28889fa18073550095/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.247315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=768d7e1ac8b24a41a1fe64761eb5249b, user_id=1e9c7afa908c4d28889fa18073550095, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:31.250315 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:31.267937 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1428/2854] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:07:31 2022] PUT /identity/v3/projects/768d7e1ac8b24a41a1fe64761eb5249b/users/1e9c7afa908c4d28889fa18073550095/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:31.279025 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e81d-6fa3-40d3-8438-b5492bec3da5 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.279488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e81d-6fa3-40d3-8438-b5492bec3da5 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.279831 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3e81d-6fa3-40d3-8438-b5492bec3da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.311804 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-fcc3e81d-6fa3-40d3-8438-b5492bec3da5 None None] MFA Rules not processed for user `1e9c7afa908c4d28889fa18073550095`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:31.313147 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-fcc3e81d-6fa3-40d3-8438-b5492bec3da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:31.400864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1427/2855] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:31.413380 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:31.415578 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:31.486691 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3e81d-6fa3-40d3-8438-b5492bec3da5 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:31.493986 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-fcc3e81d-6fa3-40d3-8438-b5492bec3da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:31.524547 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:31.536332 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3256fb0-7ac1-4bb1-afc1-ca18b438f56a admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:31.547593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675690-e57a-44cb-b084-2d76ef28ba47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:31.549458 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675690-e57a-44cb-b084-2d76ef28ba47 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.550195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675690-e57a-44cb-b084-2d76ef28ba47 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.551102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca675690-e57a-44cb-b084-2d76ef28ba47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.565165 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675690-e57a-44cb-b084-2d76ef28ba47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:31.566818 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca675690-e57a-44cb-b084-2d76ef28ba47 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:31.578991 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1429/2856] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:31.601258 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675690-e57a-44cb-b084-2d76ef28ba47 service cinder] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:31.634350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3e81d-6fa3-40d3-8438-b5492bec3da5 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:31.641648 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaa9b1-3282-4d96-89b9-2b6ce827e675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:31.643209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaa9b1-3282-4d96-89b9-2b6ce827e675 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.643767 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaa9b1-3282-4d96-89b9-2b6ce827e675 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.644050 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaa9b1-3282-4d96-89b9-2b6ce827e675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.646114 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-afaaa9b1-3282-4d96-89b9-2b6ce827e675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:31.665281 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaa9b1-3282-4d96-89b9-2b6ce827e675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:31.667074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaa9b1-3282-4d96-89b9-2b6ce827e675 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:31.678640 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca675690-e57a-44cb-b084-2d76ef28ba47 service cinder] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:31.682717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1428/2857] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 21 16:07:31.684314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:31.685409 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.685746 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.686061 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] PATH_INFO: `/v3/users/1e9c7afa908c4d28889fa18073550095` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.687326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e9c7afa908c4d28889fa18073550095)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:31.689294 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:31.766237 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e9c7afa908c4d28889fa18073550095'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:31.791511 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e9c7afa908c4d28889fa18073550095'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:31.806404 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e9c7afa908c4d28889fa18073550095'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:31.818800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e9c7afa908c4d28889fa18073550095'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:31.819567 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e9c7afa908c4d28889fa18073550095'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:31.826925 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1430/2858] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:07:31 2022] DELETE /identity/v3/users/1e9c7afa908c4d28889fa18073550095 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:07:31.831806 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a245ad-5a07-4cff-9ef8-d2da478d5408 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:31.832366 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a245ad-5a07-4cff-9ef8-d2da478d5408 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:31.832487 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a245ad-5a07-4cff-9ef8-d2da478d5408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:31.851178 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e0a245ad-5a07-4cff-9ef8-d2da478d5408 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:31.852915 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e0a245ad-5a07-4cff-9ef8-d2da478d5408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:31.871260 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:31.920540 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1429/2859] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:31.931469 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:31.933028 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:32.001007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6caa64-d5b1-46da-9526-7439e5ffc3c5 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:32.005150 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedd673-fe67-4c45-a3c9-12fdd5213dc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:32.005857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedd673-fe67-4c45-a3c9-12fdd5213dc0 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:32.006142 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedd673-fe67-4c45-a3c9-12fdd5213dc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:32.006423 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedd673-fe67-4c45-a3c9-12fdd5213dc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:32.016911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedd673-fe67-4c45-a3c9-12fdd5213dc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:32.018448 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedd673-fe67-4c45-a3c9-12fdd5213dc0 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:32.028869 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1431/2860] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:32.454055 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a245ad-5a07-4cff-9ef8-d2da478d5408 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:32.455883 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e0a245ad-5a07-4cff-9ef8-d2da478d5408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:32.471790 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a245ad-5a07-4cff-9ef8-d2da478d5408 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:32.478209 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:32.479236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:32.479529 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:32.479840 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] PATH_INFO: `/v3/projects/768d7e1ac8b24a41a1fe64761eb5249b` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:32.481832 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=768d7e1ac8b24a41a1fe64761eb5249b)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:32.483096 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:32.541172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '768d7e1ac8b24a41a1fe64761eb5249b', 'request_id': 'req-196fd69e-3cff-4d10-a2d1-2b73b636c991'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:32.552571 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '768d7e1ac8b24a41a1fe64761eb5249b', 'request_id': 'req-196fd69e-3cff-4d10-a2d1-2b73b636c991'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:32.560640 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '768d7e1ac8b24a41a1fe64761eb5249b', 'request_id': 'req-196fd69e-3cff-4d10-a2d1-2b73b636c991'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:32.573815 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] The token cache is being invalidate because project 768d7e1ac8b24a41a1fe64761eb5249b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:07:32.574109 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:32.574856 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-196fd69e-3cff-4d10-a2d1-2b73b636c991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:32.576566 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1430/2861] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:07:32 2022] DELETE /identity/v3/projects/768d7e1ac8b24a41a1fe64761eb5249b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:32.582360 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea33679-2b8e-482f-9177-a67405a6b6af None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:32.582683 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea33679-2b8e-482f-9177-a67405a6b6af None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:32.582959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea33679-2b8e-482f-9177-a67405a6b6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:32.601344 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-1ea33679-2b8e-482f-9177-a67405a6b6af None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:32.602932 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-1ea33679-2b8e-482f-9177-a67405a6b6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:32.608933 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:32.682601 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1432/2862] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:32.751446 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821b1dd-b2b4-4bb8-a562-bf743dd36f34 None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:32.751695 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821b1dd-b2b4-4bb8-a562-bf743dd36f34 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:32.752046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821b1dd-b2b4-4bb8-a562-bf743dd36f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:32.773175 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-e821b1dd-b2b4-4bb8-a562-bf743dd36f34 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:32.774454 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-e821b1dd-b2b4-4bb8-a562-bf743dd36f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:32.783350 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1431/2863] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:32.787766 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea33679-2b8e-482f-9177-a67405a6b6af None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:32.798312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea33679-2b8e-482f-9177-a67405a6b6af None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:32.803878 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:32.804480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:32.804897 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:32.805179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:32.805941 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86657) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 21 16:07:32.806249 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:32.807894 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:32.822403 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1433/2864] 10.222.0.52 () {58 vars in 1152 bytes} [Tue Jun 21 16:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 21 16:07:32.827000 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821b1dd-b2b4-4bb8-a562-bf743dd36f34 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:32.837370 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821b1dd-b2b4-4bb8-a562-bf743dd36f34 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:32.843162 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c355-a808-46a4-9df4-b07b4d37212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:32.844056 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c355-a808-46a4-9df4-b07b4d37212d admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:32.844337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c355-a808-46a4-9df4-b07b4d37212d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:32.844605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c355-a808-46a4-9df4-b07b4d37212d admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:32.845306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88c355-a808-46a4-9df4-b07b4d37212d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:32.847409 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88c355-a808-46a4-9df4-b07b4d37212d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:32.882937 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1432/2865] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:07:32 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:32.887935 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:32.898945 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50e16c-5a03-4fa7-917f-3a419b23eee4 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:32.960347 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f355c9e-3279-4628-bf35-17ec842c2860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:32.961086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f355c9e-3279-4628-bf35-17ec842c2860 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:32.961333 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f355c9e-3279-4628-bf35-17ec842c2860 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:32.961618 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f355c9e-3279-4628-bf35-17ec842c2860 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:32.962336 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f355c9e-3279-4628-bf35-17ec842c2860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:32.964279 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f355c9e-3279-4628-bf35-17ec842c2860 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:32.996311 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1434/2866] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:07:32 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:33.000896 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c355-a808-46a4-9df4-b07b4d37212d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.018604 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c355-a808-46a4-9df4-b07b4d37212d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.024231 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2a48f-5989-44b5-8186-03be5f40df3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.024919 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a48f-5989-44b5-8186-03be5f40df3c admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.025208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a48f-5989-44b5-8186-03be5f40df3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.025481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2a48f-5989-44b5-8186-03be5f40df3c admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.026319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2a48f-5989-44b5-8186-03be5f40df3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.027940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2a48f-5989-44b5-8186-03be5f40df3c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.040004 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1433/2867] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:07:33.044215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f355c9e-3279-4628-bf35-17ec842c2860 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.056138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f355c9e-3279-4628-bf35-17ec842c2860 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.062065 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b485725-9f18-4a67-9696-512621688ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.062702 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b485725-9f18-4a67-9696-512621688ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.062982 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b485725-9f18-4a67-9696-512621688ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.063254 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b485725-9f18-4a67-9696-512621688ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.063984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b485725-9f18-4a67-9696-512621688ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.065789 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b485725-9f18-4a67-9696-512621688ab0 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.080701 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1435/2868] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:07:33.082488 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2a48f-5989-44b5-8186-03be5f40df3c admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.093469 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2a48f-5989-44b5-8186-03be5f40df3c admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.101115 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfd0ce-45c6-4659-a2e9-c300597af3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.102900 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfd0ce-45c6-4659-a2e9-c300597af3e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.103364 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfd0ce-45c6-4659-a2e9-c300597af3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.103845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfd0ce-45c6-4659-a2e9-c300597af3e0 admin admin] PATH_INFO: `/v3/projects/bf866ef186ae476ca43487dbeab93bdd/users/4ef36a91cebf4139bf0c57bbc0effaf9/roles/ff19e0e651ce4a5a84fa63e1b3f386a5` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.128427 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfd0ce-45c6-4659-a2e9-c300597af3e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf866ef186ae476ca43487dbeab93bdd, user_id=4ef36a91cebf4139bf0c57bbc0effaf9, role_id=ff19e0e651ce4a5a84fa63e1b3f386a5)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.131299 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfd0ce-45c6-4659-a2e9-c300597af3e0 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.146967 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1434/2869] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:07:33 2022] PUT /identity/v3/projects/bf866ef186ae476ca43487dbeab93bdd/users/4ef36a91cebf4139bf0c57bbc0effaf9/roles/ff19e0e651ce4a5a84fa63e1b3f386a5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:33.157537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0805b5b-9761-4c4b-b947-1265e045fe26 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.157857 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0805b5b-9761-4c4b-b947-1265e045fe26 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.158143 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0805b5b-9761-4c4b-b947-1265e045fe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.187394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-d0805b5b-9761-4c4b-b947-1265e045fe26 None None] MFA Rules not processed for user `4ef36a91cebf4139bf0c57bbc0effaf9`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:33.188657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-d0805b5b-9761-4c4b-b947-1265e045fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:33.264608 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1436/2870] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 21 16:07:33.269661 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfd0ce-45c6-4659-a2e9-c300597af3e0 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.321671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:33.346658 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfd0ce-45c6-4659-a2e9-c300597af3e0 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.353036 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c15f94-7926-4efb-9cbf-f80a6c0f971d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.353734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c15f94-7926-4efb-9cbf-f80a6c0f971d admin admin] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.354046 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c15f94-7926-4efb-9cbf-f80a6c0f971d admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.354319 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c15f94-7926-4efb-9cbf-f80a6c0f971d admin admin] PATH_INFO: `/v3/projects` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.355040 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c15f94-7926-4efb-9cbf-f80a6c0f971d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.356973 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c15f94-7926-4efb-9cbf-f80a6c0f971d admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.388899 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1435/2871] 10.222.0.52 () {60 vars in 1152 bytes} [Tue Jun 21 16:07:33 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:33.393927 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0805b5b-9761-4c4b-b947-1265e045fe26 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.409396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0805b5b-9761-4c4b-b947-1265e045fe26 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.414310 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:33.476078 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f9a9e-0724-44dd-9832-d448b650f457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.476757 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f9a9e-0724-44dd-9832-d448b650f457 admin admin] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.477040 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f9a9e-0724-44dd-9832-d448b650f457 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.477312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f9a9e-0724-44dd-9832-d448b650f457 admin admin] PATH_INFO: `/v3/users` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.478400 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f9a9e-0724-44dd-9832-d448b650f457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.480326 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f9a9e-0724-44dd-9832-d448b650f457 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.503955 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1437/2872] 10.222.0.52 () {60 vars in 1143 bytes} [Tue Jun 21 16:07:33 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 21 16:07:33.507965 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c15f94-7926-4efb-9cbf-f80a6c0f971d admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.519174 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c15f94-7926-4efb-9cbf-f80a6c0f971d admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.524854 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f32f0-ee58-47cd-839e-a6235d8ced95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.525629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f32f0-ee58-47cd-839e-a6235d8ced95 admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.526015 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f32f0-ee58-47cd-839e-a6235d8ced95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.526361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f32f0-ee58-47cd-839e-a6235d8ced95 admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.527208 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f32f0-ee58-47cd-839e-a6235d8ced95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.529110 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f32f0-ee58-47cd-839e-a6235d8ced95 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.542734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1436/2873] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 21 16:07:33.546903 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f9a9e-0724-44dd-9832-d448b650f457 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.558088 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f9a9e-0724-44dd-9832-d448b650f457 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.564491 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe545-5de5-4557-9333-050ba3c21496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.566429 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe545-5de5-4557-9333-050ba3c21496 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.566898 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe545-5de5-4557-9333-050ba3c21496 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.567250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffe545-5de5-4557-9333-050ba3c21496 admin admin] PATH_INFO: `/v3/projects/4854a7fccb184b45bf1e08935eaf3d85/users/eef0b82e42564adea7fbc7b6c91095de/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.587080 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffe545-5de5-4557-9333-050ba3c21496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4854a7fccb184b45bf1e08935eaf3d85, user_id=eef0b82e42564adea7fbc7b6c91095de, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.589764 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffe545-5de5-4557-9333-050ba3c21496 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.605263 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1438/2874] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:07:33 2022] PUT /identity/v3/projects/4854a7fccb184b45bf1e08935eaf3d85/users/eef0b82e42564adea7fbc7b6c91095de/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:33.612777 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f32f0-ee58-47cd-839e-a6235d8ced95 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.624043 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f32f0-ee58-47cd-839e-a6235d8ced95 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.687322 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0248be3-724c-44ef-9835-bb19e48e364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.687992 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248be3-724c-44ef-9835-bb19e48e364c admin admin] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.688337 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248be3-724c-44ef-9835-bb19e48e364c admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.688576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248be3-724c-44ef-9835-bb19e48e364c admin admin] PATH_INFO: `/v3/roles` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.689355 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0248be3-724c-44ef-9835-bb19e48e364c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.691236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0248be3-724c-44ef-9835-bb19e48e364c admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.706473 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1437/2875] 10.222.0.52 () {58 vars in 1121 bytes} [Tue Jun 21 16:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 21 16:07:33.711023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe545-5de5-4557-9333-050ba3c21496 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.721729 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffe545-5de5-4557-9333-050ba3c21496 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.726829 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.728090 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] REQUEST_METHOD: `PUT` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.728362 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.728633 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] PATH_INFO: `/v3/projects/4854a7fccb184b45bf1e08935eaf3d85/users/eef0b82e42564adea7fbc7b6c91095de/roles/caaf25fef38b4e7e8601fe46da291ce8` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.731017 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4854a7fccb184b45bf1e08935eaf3d85, user_id=eef0b82e42564adea7fbc7b6c91095de, role_id=caaf25fef38b4e7e8601fe46da291ce8)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:33.733731 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:33.745643 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1439/2876] 10.222.0.52 () {60 vars in 1482 bytes} [Tue Jun 21 16:07:33 2022] PUT /identity/v3/projects/4854a7fccb184b45bf1e08935eaf3d85/users/eef0b82e42564adea7fbc7b6c91095de/roles/caaf25fef38b4e7e8601fe46da291ce8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:07:33.757520 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360696d-a008-4bfd-88b9-e80c2de1fcdf None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.758089 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360696d-a008-4bfd-88b9-e80c2de1fcdf None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.758599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360696d-a008-4bfd-88b9-e80c2de1fcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:33.795137 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-0360696d-a008-4bfd-88b9-e80c2de1fcdf None None] MFA Rules not processed for user `eef0b82e42564adea7fbc7b6c91095de`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:33.796447 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-0360696d-a008-4bfd-88b9-e80c2de1fcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:33.878859 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1438/2877] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:33.891215 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:33.893712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:33.957051 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c2bed-3e9f-4a4f-80ec-9ede4a45e551 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:33.961086 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6a135-2869-43f2-9106-c9d855d767e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:33.961744 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6a135-2869-43f2-9106-c9d855d767e1 service cinder] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:33.962069 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6a135-2869-43f2-9106-c9d855d767e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:33.962350 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6a135-2869-43f2-9106-c9d855d767e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:34.034181 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6a135-2869-43f2-9106-c9d855d767e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:34.035805 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6a135-2869-43f2-9106-c9d855d767e1 service cinder] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:34.045881 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1440/2878] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:36.635857 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360696d-a008-4bfd-88b9-e80c2de1fcdf None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:36.679246 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:36.703448 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360696d-a008-4bfd-88b9-e80c2de1fcdf None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:36.709386 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:36.710304 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:36.710582 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:36.710853 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] PATH_INFO: `/v3/users/4ef36a91cebf4139bf0c57bbc0effaf9` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:36.712307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef36a91cebf4139bf0c57bbc0effaf9)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:36.714279 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:36.763931 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef36a91cebf4139bf0c57bbc0effaf9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:36.776060 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef36a91cebf4139bf0c57bbc0effaf9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:36.784877 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef36a91cebf4139bf0c57bbc0effaf9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:36.785694 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef36a91cebf4139bf0c57bbc0effaf9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:36.792920 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef36a91cebf4139bf0c57bbc0effaf9'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:36.799192 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1439/2879] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:07:36 2022] DELETE /identity/v3/users/4ef36a91cebf4139bf0c57bbc0effaf9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:36.804931 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a4c07-a466-40c5-ba84-826a5bf271d1 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:36.805845 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a4c07-a466-40c5-ba84-826a5bf271d1 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:36.806027 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a4c07-a466-40c5-ba84-826a5bf271d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:36.827312 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-312a4c07-a466-40c5-ba84-826a5bf271d1 None None] MFA Rules not processed for user `945ae273f8ef413cad49cd7331140c10`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:07:36.828512 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-312a4c07-a466-40c5-ba84-826a5bf271d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:36.840103 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:36.898140 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1441/2880] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:07:36.908106 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:36.909714 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:36.986864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928df27-774c-4660-b209-3668b7f19975 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:36.992395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbfcf0-3b0e-4753-83ad-70530e72d248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:36.993072 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfcf0-3b0e-4753-83ad-70530e72d248 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:36.993360 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfcf0-3b0e-4753-83ad-70530e72d248 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:36.993678 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbfcf0-3b0e-4753-83ad-70530e72d248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:37.004475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbfcf0-3b0e-4753-83ad-70530e72d248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:37.005996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cbfcf0-3b0e-4753-83ad-70530e72d248 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:37.016314 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1440/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:37.389911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a4c07-a466-40c5-ba84-826a5bf271d1 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:37.391452 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-312a4c07-a466-40c5-ba84-826a5bf271d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:37.409219 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a4c07-a466-40c5-ba84-826a5bf271d1 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:37.415081 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:37.416033 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:37.416283 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:37.416558 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] PATH_INFO: `/v3/projects/bf866ef186ae476ca43487dbeab93bdd` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:37.419228 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf866ef186ae476ca43487dbeab93bdd)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:37.421208 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:37.472032 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf866ef186ae476ca43487dbeab93bdd', 'request_id': 'req-61936631-aa08-4c1f-a606-412329930c5c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.478740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf866ef186ae476ca43487dbeab93bdd', 'request_id': 'req-61936631-aa08-4c1f-a606-412329930c5c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.488101 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf866ef186ae476ca43487dbeab93bdd', 'request_id': 'req-61936631-aa08-4c1f-a606-412329930c5c'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.499632 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] The token cache is being invalidate because project bf866ef186ae476ca43487dbeab93bdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:07:37.499924 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.500684 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.502393 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1442/2882] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:07:37 2022] DELETE /identity/v3/projects/bf866ef186ae476ca43487dbeab93bdd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:37.508116 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbfcf0-3b0e-4753-83ad-70530e72d248 service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:37.518426 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbfcf0-3b0e-4753-83ad-70530e72d248 service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:37.585573 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:37.586507 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:37.586787 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:37.587051 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] PATH_INFO: `/v3/users/eef0b82e42564adea7fbc7b6c91095de` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:37.588272 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef0b82e42564adea7fbc7b6c91095de)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:37.590265 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:37.591735 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:37.644156 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef0b82e42564adea7fbc7b6c91095de'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.657223 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef0b82e42564adea7fbc7b6c91095de'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.666948 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef0b82e42564adea7fbc7b6c91095de'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.667508 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef0b82e42564adea7fbc7b6c91095de'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.674830 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef0b82e42564adea7fbc7b6c91095de'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:37.681236 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1441/2883] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:07:37 2022] DELETE /identity/v3/users/eef0b82e42564adea7fbc7b6c91095de => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:37.978352 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:37.979905 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:37.996966 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61936631-aa08-4c1f-a606-412329930c5c admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:38.059316 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:38.060202 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:38.060486 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:38.060878 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] PATH_INFO: `/v3/projects/4854a7fccb184b45bf1e08935eaf3d85` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:38.062104 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4854a7fccb184b45bf1e08935eaf3d85)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:38.064029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:38.115063 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:38.125713 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4854a7fccb184b45bf1e08935eaf3d85', 'request_id': 'req-0eaa13b1-dc2a-447a-95fb-a2d17496da84'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:38.135394 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4854a7fccb184b45bf1e08935eaf3d85', 'request_id': 'req-0eaa13b1-dc2a-447a-95fb-a2d17496da84'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:38.146819 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4854a7fccb184b45bf1e08935eaf3d85', 'request_id': 'req-0eaa13b1-dc2a-447a-95fb-a2d17496da84'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:38.160223 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] The token cache is being invalidate because project 4854a7fccb184b45bf1e08935eaf3d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:07:38.160957 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:38.161677 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:07:38.163396 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1443/2884] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:07:37 2022] DELETE /identity/v3/projects/4854a7fccb184b45bf1e08935eaf3d85 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:07:42.365307 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:42.367367 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:42.450842 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40345f5-4974-4f72-9b46-4d24a0eae204 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:42.518172 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:42.518738 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:42.519024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:42.519335 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:42.521285 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:42.534817 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:42.536546 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:42.547671 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1442/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:45.734010 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:07:45.735728 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:45.809008 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa13b1-dc2a-447a-95fb-a2d17496da84 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:07:45.814808 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c46e0-5c30-4914-a53f-bc66266050b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:07:45.815839 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c46e0-5c30-4914-a53f-bc66266050b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:07:45.816295 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c46e0-5c30-4914-a53f-bc66266050b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:07:45.816734 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c46e0-5c30-4914-a53f-bc66266050b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:07:45.820268 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-464c46e0-5c30-4914-a53f-bc66266050b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:07:45.887614 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c46e0-5c30-4914-a53f-bc66266050b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:07:45.889073 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c46e0-5c30-4914-a53f-bc66266050b2 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:07:45.901436 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1444/2886] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:07:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:07:52.543044 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:07:55.895300 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=60,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:08:27.300576 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:08:27.302356 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:08:27.363517 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c633892-8513-45dc-8649-ca478bc065a9 service nova] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:08:27.368492 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:08:27.369152 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:08:27.369437 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:08:27.369706 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:08:27.371720 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:08:27.433521 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:08:27.435019 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:08:27.444527 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1443/2887] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:08:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:08:37.439988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=48 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:08:37.503307 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e3969-777d-4e67-9369-54d01287656f None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:08:37.503860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e3969-777d-4e67-9369-54d01287656f None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:08:37.503860 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e3969-777d-4e67-9369-54d01287656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:08:37.521672 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-b62e3969-777d-4e67-9369-54d01287656f None None] MFA Rules not processed for user `0943474b3d4d49d7973619388dab2be1`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:08:37.523411 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-b62e3969-777d-4e67-9369-54d01287656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:08:37.531707 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1445/2888] 10.222.0.52 () {58 vars in 957 bytes} [Tue Jun 21 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:08:37.570292 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:08:37.571892 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:08:37.584040 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd942c-2a91-4c2a-b31d-2e6318e13973 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:08:37.588541 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed0934c0b1a64dfca3431dc32441e2db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:08:37.589197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:08:37.589481 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:08:37.589750 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:08:37.591980 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:08:37.619923 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:08:37.621379 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:08:37.642910 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1444/2889] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:08:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:08:38.438615 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e3969-777d-4e67-9369-54d01287656f None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:08:38.449419 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e3969-777d-4e67-9369-54d01287656f None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:08:38.454024 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:08:38.454952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:08:38.455230 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:08:38.455503 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:08:38.457357 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:08:38.474751 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:08:38.476196 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:08:38.489286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1446/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:08:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:08:47.639863 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=15 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:08:48.484243 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=9 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:08:54.497024 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:08:54.511390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70a0f8-7eeb-4183-a426-d189b6585877 service cinder] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:08:54.515732 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e565c39-1d52-4866-a5af-03f6fe16422c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:08:54.516390 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565c39-1d52-4866-a5af-03f6fe16422c service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:08:54.516672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565c39-1d52-4866-a5af-03f6fe16422c service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:08:54.516972 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565c39-1d52-4866-a5af-03f6fe16422c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:08:54.519675 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-5e565c39-1d52-4866-a5af-03f6fe16422c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:08:54.592214 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e565c39-1d52-4866-a5af-03f6fe16422c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:08:54.593700 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e565c39-1d52-4866-a5af-03f6fe16422c service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:08:54.604660 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1445/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:08:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:09:04.599711 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=30 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:09:45.694939 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:09:45.704314 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e333e85-aa78-4f14-98a5-c985b3475151 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:09:45.709670 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f2d4e-4273-4972-a0eb-ddac76a6693d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:09:45.710384 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2d4e-4273-4972-a0eb-ddac76a6693d service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:09:45.710722 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2d4e-4273-4972-a0eb-ddac76a6693d service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:09:45.710949 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2d4e-4273-4972-a0eb-ddac76a6693d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:09:45.714717 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-2f6f2d4e-4273-4972-a0eb-ddac76a6693d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:09:45.731437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f2d4e-4273-4972-a0eb-ddac76a6693d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:09:45.732900 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f2d4e-4273-4972-a0eb-ddac76a6693d service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:09:45.742042 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1447/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:09:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:09:55.737666 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=6 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:10:25.709670 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b683-b46e-49be-af69-c2f21eecd461 None None] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:10:25.710308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b683-b46e-49be-af69-c2f21eecd461 None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:10:25.710308 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b683-b46e-49be-af69-c2f21eecd461 None None] PATH_INFO: `/` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:10:25.711669 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1446/2893] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 16:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 16:10:25.718132 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ce8d5-122d-4f86-a91e-92c3764f3539 None None] REQUEST_METHOD: `POST` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:10:25.719521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ce8d5-122d-4f86-a91e-92c3764f3539 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:10:25.719521 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ce8d5-122d-4f86-a91e-92c3764f3539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:10:25.783003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.auth.core [None req-c99ce8d5-122d-4f86-a91e-92c3764f3539 None None] MFA Rules not processed for user `6efa4bed025741d492158b5c6e45c7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86657) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:10:25.784549 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-c99ce8d5-122d-4f86-a91e-92c3764f3539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:10:25.792007 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1448/2894] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 16:10:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:10:25.806001 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094b683-b46e-49be-af69-c2f21eecd461 None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:10:25.807908 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-6094b683-b46e-49be-af69-c2f21eecd461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:10:25.822071 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094b683-b46e-49be-af69-c2f21eecd461 None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:10:25.825717 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804ca1a-cda0-4de5-b96b-b1325f8244a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:10:25.826392 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804ca1a-cda0-4de5-b96b-b1325f8244a5 service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:10:25.826672 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804ca1a-cda0-4de5-b96b-b1325f8244a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:10:25.826955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804ca1a-cda0-4de5-b96b-b1325f8244a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:10:25.835126 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804ca1a-cda0-4de5-b96b-b1325f8244a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:10:25.836585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804ca1a-cda0-4de5-b96b-b1325f8244a5 service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:10:25.844734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1447/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:10:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:10:26.938630 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046ca5-ba05-46ac-bf77-e272f1bd1041 None None] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:10:26.939222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046ca5-ba05-46ac-bf77-e272f1bd1041 None None] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:10:26.939222 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046ca5-ba05-46ac-bf77-e272f1bd1041 None None] PATH_INFO: `/` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:10:26.940671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1449/2896] 10.222.0.52 () {54 vars in 921 bytes} [Tue Jun 21 16:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 21 16:10:26.947617 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c332768-bd42-461a-be8e-f693e52812cc None None] REQUEST_METHOD: `POST` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:10:26.948018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c332768-bd42-461a-be8e-f693e52812cc None None] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:10:26.948018 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c332768-bd42-461a-be8e-f693e52812cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:10:26.989559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.auth.core [None req-4c332768-bd42-461a-be8e-f693e52812cc None None] MFA Rules not processed for user `46e5ca80b7834ab7bc951b454c9aae49`. Rule list: `[]` (Enabled: `True`). {{(pid=86658) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 21 16:10:26.989850 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4c332768-bd42-461a-be8e-f693e52812cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:10:26.999618 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1448/2897] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Jun 21 16:10:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 21 16:10:27.010275 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046ca5-ba05-46ac-bf77-e272f1bd1041 None None] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:10:27.023853 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046ca5-ba05-46ac-bf77-e272f1bd1041 None None] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:10:27.027741 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc375d1a-dfd1-4e59-aa1a-19e252118924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:10:27.028484 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc375d1a-dfd1-4e59-aa1a-19e252118924 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:10:27.028834 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc375d1a-dfd1-4e59-aa1a-19e252118924 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:10:27.029179 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc375d1a-dfd1-4e59-aa1a-19e252118924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:10:27.040911 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc375d1a-dfd1-4e59-aa1a-19e252118924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:10:27.042496 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc375d1a-dfd1-4e59-aa1a-19e252118924 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:10:27.054182 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1450/2898] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:10:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:10:36.985663 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=15 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:10:37.048910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=19 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:10:44.358734 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c332768-bd42-461a-be8e-f693e52812cc None None] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:10:44.360458 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4c332768-bd42-461a-be8e-f693e52812cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:10:44.376034 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c332768-bd42-461a-be8e-f693e52812cc None None] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:10:44.382599 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46e5ca80b7834ab7bc951b454c9aae49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:10:44.383276 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:10:44.383562 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:10:44.383845 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:10:44.385712 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:10:44.398941 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:10:44.400552 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:10:44.414794 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1449/2899] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:10:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:10:54.410866 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=6 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:10:55.955646 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc375d1a-dfd1-4e59-aa1a-19e252118924 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:10:55.965736 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc375d1a-dfd1-4e59-aa1a-19e252118924 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:10:55.969934 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:10:55.970599 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:10:55.970867 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:10:55.971136 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:10:55.973767 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:10:56.042495 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:10:56.044006 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:10:56.055130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1451/2900] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:10:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:11:02.489559 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:11:02.491233 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:11:02.553335 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:11:02.555993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:11:02.572657 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3878e4be-9a51-4116-b132-4eff315fec34 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:11:02.577519 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:11:02.578428 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:11:02.578712 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:11:02.578984 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] PATH_INFO: `/v3/users/0943474b3d4d49d7973619388dab2be1` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:11:02.580186 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0943474b3d4d49d7973619388dab2be1)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:11:02.581958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8986ab-c650-4864-8a8a-6206dfc938e1 service nova] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:11:02.582288 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:11:02.587411 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:11:02.588074 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:11:02.588358 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:11:02.588637 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:11:02.590475 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:11:02.603940 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:11:02.606428 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:11:02.618807 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1450/2901] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:11:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:11:02.638856 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0943474b3d4d49d7973619388dab2be1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:02.653693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0943474b3d4d49d7973619388dab2be1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:02.666003 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0943474b3d4d49d7973619388dab2be1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:02.673688 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0943474b3d4d49d7973619388dab2be1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:02.674671 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0943474b3d4d49d7973619388dab2be1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:02.680938 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1452/2902] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:11:02 2022] DELETE /identity/v3/users/0943474b3d4d49d7973619388dab2be1 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 21 16:11:02.864345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:11:02.877373 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836c174-39c1-4e36-ae63-cf043400886a service placement] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:11:02.946345 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:11:02.947316 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:11:02.947605 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:11:02.947955 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] PATH_INFO: `/v3/projects/658a35680c70498db2bccc1085c22b21` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:11:02.949301 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=658a35680c70498db2bccc1085c22b21)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:11:02.951402 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:11:03.002138 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658a35680c70498db2bccc1085c22b21', 'request_id': 'req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.011763 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658a35680c70498db2bccc1085c22b21', 'request_id': 'req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.023194 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658a35680c70498db2bccc1085c22b21', 'request_id': 'req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.036007 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] The token cache is being invalidate because project 658a35680c70498db2bccc1085c22b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:11:03.036295 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.036998 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.038643 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1451/2903] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:11:02 2022] DELETE /identity/v3/projects/658a35680c70498db2bccc1085c22b21 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:11:03.042651 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:11:03.044053 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:11:03.059536 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da559b-1c16-4d86-89df-9a019d869ce8 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:11:03.120137 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:11:03.121107 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:11:03.121397 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:11:03.121675 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] PATH_INFO: `/v3/users/0b8ab25c4b3b435f86b85c219a908f71` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:11:03.122962 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8ab25c4b3b435f86b85c219a908f71)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:11:03.124959 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:11:03.131480 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:11:03.177233 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8ab25c4b3b435f86b85c219a908f71'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.189446 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8ab25c4b3b435f86b85c219a908f71'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.200609 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8ab25c4b3b435f86b85c219a908f71'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.208833 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8ab25c4b3b435f86b85c219a908f71'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.209659 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8ab25c4b3b435f86b85c219a908f71'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.216437 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1453/2904] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:11:03 2022] DELETE /identity/v3/users/0b8ab25c4b3b435f86b85c219a908f71 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:11:03.410635 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:11:03.412445 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:11:03.426997 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c447b1-befd-40b9-bca0-6c3f1cc61bfa admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:11:03.446258 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:11:03.490092 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:11:03.490958 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:11:03.491264 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:11:03.491534 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] PATH_INFO: `/v3/projects/983d1968d3a24c1882a8973cbd495e36` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:11:03.492730 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=983d1968d3a24c1882a8973cbd495e36)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:11:03.494680 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:11:03.543197 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983d1968d3a24c1882a8973cbd495e36', 'request_id': 'req-71d8d707-4646-4adb-b9ce-82d4bc44f5df'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.552347 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983d1968d3a24c1882a8973cbd495e36', 'request_id': 'req-71d8d707-4646-4adb-b9ce-82d4bc44f5df'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.558864 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983d1968d3a24c1882a8973cbd495e36', 'request_id': 'req-71d8d707-4646-4adb-b9ce-82d4bc44f5df'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.570091 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] The token cache is being invalidate because project 983d1968d3a24c1882a8973cbd495e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86658) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:11:03.570380 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.571117 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:11:03.572802 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1452/2905] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:11:03 2022] DELETE /identity/v3/projects/983d1968d3a24c1882a8973cbd495e36 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:11:13.212544 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:11:13.566758 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:11:51.930593 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:11:51.932426 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:11:52.016023 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a130f-0fe2-4828-82ff-5967ad376a08 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:11:52.079616 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:11:52.080515 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:11:52.080740 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:11:52.080952 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:11:52.083029 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:11:52.094993 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:11:52.096735 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:11:52.107169 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1454/2906] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:11:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:12:02.102591 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=58 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:12:12.374149 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:12:12.375852 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:12:12.436944 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8d707-4646-4adb-b9ce-82d4bc44f5df admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:12:12.441287 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:12:12.441978 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:12:12.442266 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:12:12.442545 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:12:12.444490 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:12:12.508002 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:12:12.509361 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:12:12.518755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1453/2907] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:12:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:12:17.258807 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:12:17.268693 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031c5e5-bf68-40db-9b7a-5edc3374a38d service placement] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:12:17.272784 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6efa4bed025741d492158b5c6e45c7e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:12:17.273434 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] REQUEST_METHOD: `GET` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:12:17.273714 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:12:17.274019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:12:17.275838 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:12:17.343804 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:12:17.345286 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:12:17.356039 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1455/2908] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:12:22.514393 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=48 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:12:23.564080 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:12:23.565587 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:12:23.596677 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97414285-9c05-46b7-82f9-88d0bf29223a service neutron] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:12:23.601988 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:12:23.602987 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:12:23.603218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:12:23.603584 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] PATH_INFO: `/v3/users/8617fdc3e63446c89fcd22bd5756109d` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:12:23.605414 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8617fdc3e63446c89fcd22bd5756109d)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:12:23.607387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:12:23.656484 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8617fdc3e63446c89fcd22bd5756109d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:23.676878 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8617fdc3e63446c89fcd22bd5756109d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:23.684803 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8617fdc3e63446c89fcd22bd5756109d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:23.685440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8617fdc3e63446c89fcd22bd5756109d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:23.692755 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8617fdc3e63446c89fcd22bd5756109d'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:23.698881 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1454/2909] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:12:23 2022] DELETE /identity/v3/users/8617fdc3e63446c89fcd22bd5756109d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:12:23.895576 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:12:23.906841 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14170dd5-a3f7-4777-84a6-f5ca17881297 service neutron] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:12:23.976493 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:12:23.977276 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:12:23.977555 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:12:23.977861 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] PATH_INFO: `/v3/projects/4016a8a7626d4aa5a28d248a5a799e8e` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:12:23.979082 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4016a8a7626d4aa5a28d248a5a799e8e)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:12:23.981290 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:12:24.030566 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4016a8a7626d4aa5a28d248a5a799e8e', 'request_id': 'req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.037130 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4016a8a7626d4aa5a28d248a5a799e8e', 'request_id': 'req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.044681 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4016a8a7626d4aa5a28d248a5a799e8e', 'request_id': 'req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.055247 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] The token cache is being invalidate because project 4016a8a7626d4aa5a28d248a5a799e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:12:24.055534 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.056195 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.057800 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1456/2910] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:12:23 2022] DELETE /identity/v3/projects/4016a8a7626d4aa5a28d248a5a799e8e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:12:24.061873 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:12:24.063306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:12:24.077397 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fffe71-7e0d-4886-a920-5f67a91aa269 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:12:24.141374 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:12:24.142306 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:12:24.142585 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:12:24.142862 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] PATH_INFO: `/v3/users/c675841a37b5419aa6d448cfa90624ed` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:12:24.144404 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c675841a37b5419aa6d448cfa90624ed)` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:12:24.146381 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:12:24.194799 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c675841a37b5419aa6d448cfa90624ed'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.206805 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c675841a37b5419aa6d448cfa90624ed'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.214580 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c675841a37b5419aa6d448cfa90624ed'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.215218 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c675841a37b5419aa6d448cfa90624ed'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.217440 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:12:24.225902 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.notifications [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c675841a37b5419aa6d448cfa90624ed'} {{(pid=86658) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.233232 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1455/2911] 10.222.0.52 () {58 vars in 1223 bytes} [Tue Jun 21 16:12:24 2022] DELETE /identity/v3/users/c675841a37b5419aa6d448cfa90624ed => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:12:24.421975 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] Authenticating user token {{(pid=86657) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:12:24.423537 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.fernet_utils [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86657) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:12:24.437138 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1b3ec-66bc-4a68-a484-83b7ebb9f3e1 admin admin] Validating token access rules against request {{(pid=86657) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:12:24.461732 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:12:24.506154 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '945ae273f8ef413cad49cd7331140c10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb46a2622cdd481281f00889cb2ab74f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86657) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:12:24.506974 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:12:24.507256 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] SCRIPT_NAME: `/identity` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:12:24.507535 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] PATH_INFO: `/v3/projects/4e019b6e83324155848189e56bdf0a4e` {{(pid=86657) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:12:24.508910 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e019b6e83324155848189e56bdf0a4e)` {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:12:24.511250 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] RBAC: Authorization granted {{(pid=86657) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:12:24.560366 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e019b6e83324155848189e56bdf0a4e', 'request_id': 'req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.567102 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e019b6e83324155848189e56bdf0a4e', 'request_id': 'req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.576019 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e019b6e83324155848189e56bdf0a4e', 'request_id': 'req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df'} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.587038 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] The token cache is being invalidate because project 4e019b6e83324155848189e56bdf0a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86657) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 21 16:12:24.587330 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.587992 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG keystone.notifications [None req-e1d797a4-fdd2-40c0-a8d2-113d5a39a1df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86657) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 21 16:12:24.589592 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: [pid: 86657|app: 0|req: 1457/2912] 10.222.0.52 () {58 vars in 1232 bytes} [Tue Jun 21 16:12:24 2022] DELETE /identity/v3/projects/4e019b6e83324155848189e56bdf0a4e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 21 16:12:34.229632 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=4 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:12:34.583948 dv-cinder-840678-13-iscsi devstack@keystone.service[86657]: DEBUG dbcounter [-] [86657] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86657) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 21 16:16:54.929022 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] Authenticating user token {{(pid=86658) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 21 16:16:54.930801 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:16:55.081140 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88413aad-1b7b-4cd2-b9a4-c571844e5b76 admin admin] Validating token access rules against request {{(pid=86658) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 21 16:16:55.142146 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c2d3e-ee2b-47eb-92f6-45d90a249da1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16defb911ffa485da9164fa2674baba1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '60408c00fa3d4698b251ca2a59ae8c5f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86658) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 21 16:16:55.142809 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c2d3e-ee2b-47eb-92f6-45d90a249da1 service placement] REQUEST_METHOD: `GET` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 21 16:16:55.143113 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c2d3e-ee2b-47eb-92f6-45d90a249da1 service placement] SCRIPT_NAME: `/identity` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 21 16:16:55.143387 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c2d3e-ee2b-47eb-92f6-45d90a249da1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86658) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 21 16:16:55.145629 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.fernet_utils [None req-d78c2d3e-ee2b-47eb-92f6-45d90a249da1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86658) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 21 16:16:55.158395 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c2d3e-ee2b-47eb-92f6-45d90a249da1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 21 16:16:55.159996 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78c2d3e-ee2b-47eb-92f6-45d90a249da1 service placement] RBAC: Authorization granted {{(pid=86658) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 21 16:16:55.170161 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: [pid: 86658|app: 0|req: 1456/2913] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Jun 21 16:16:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 21 16:17:05.166069 dv-cinder-840678-13-iscsi devstack@keystone.service[86658]: DEBUG dbcounter [-] [86658] Writing DB stats keystone:SELECT=89 {{(pid=86658) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}